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PREFACE 


The  Logistics  Research  Division  of  Armstrong  Laboratory  (A1VHRG)  held  a  work¬ 
shop  in  Palo  Alto,  California,  and  a  follow-up  session  at  Wright-Patterson  Air  Force  Base 
to  explore  emerging  technologies  which  could  be  applied  to  the  automated  production  and 
presentation  of  aircraft  maintenance  information.  The  overall  goal  of  the  workshops  and 
this  report  is  to  help  establish  a  research  agenda  for  AL/HRG  in  advancing  the  coming  gen¬ 
eration  of  Interactive  Electronic  Technical  Manuals  (IETMs).  This  will  not  only  serve 
Armstrong  Laboratory  in  project  planning  but  will  also  provide  a  general  overview,  along 
with  specific  guidelines,  for  prospective  research  and  development  teams. 

The  ideas  presented  in  this  report  were  drawn,  over  the  course  of  the  workshops,  from 
a  panel  of  experts  in  various  applicable  Fields.  These  include  original  engineering  design 
extraction,  qualitative  reasoning,  human-modeling,  automated  task  planning,  generation  of 
multimedia  explanations,  and  user  interface  technology  (including  virtual  reality).  Research 
addressing  short-,  medium-,  and  long-term  goals  is  discussed  in  each  of  these  areas,  as 
well  as  in  emerging  technologies  which  cut  across  the  various  phases  of  technical  manual 
production,  such  as  automated  consistency  checking  and  automated  validation  and  verifica¬ 
tion. 


These  workshops  would  not  have  been  possible  without  the  innovative  ideas,  detailed 
planning,  and  overall  support  of  David  Gunning  at  Armstrong  Laboratory.  Dave’s  vision 
and  determination  continue  to  be  an  inspiration  to  all  those  involved  in  the  future  of  Air 
Force  technical  manuals.  The  author  also  acknowledges  the  influence  of  AL/HRG  division 
chief,  Bertram  W.  Cream,  and  Robert  C.  Johnson,  chief  of  the  Operational  Logistics 
Branch,  along  with  the  rest  of  the  staff  at  AL/HRG,  for  their  assistance  in  making  these 
workshops  happen. 

The  research  views  presented  here  are  drawn  directly  from  those  of  the  distinguished 
participants.  The  author  gratefully  credits  these  experts  for  their  ideas  and  assessments, 
which  were  collected  through  the  scheduled  presentations,  individual  discussions  during 
the  workshops,  and  various  notes  and  suggestions  provided  afterwards.  Bert  Dimock,  of 
Dimock  Associates,  provided  much  of  the  introductory  material  appearing  in  this  report  and 
helped  prepare  the  initial  drafts.  The  technical  contributions  of  Norman  Badler,  Ed  Boyle, 
Mark  Drummond,  Steven  Feiner,  and  Gary  Worrall  deserve  particular  recognition  as  well. 

Thanks  go  to  Mark  Miller  and  Garth  Cooke,  of  Systems  Exploration  Incorporated,  for 
competent  handling  of  local  arrangements.  Jeff  Kitson  and  Maria  Pryce  of  Kestrel  Institute 
helped  with  filming  and  various  “last  minute”  details  during  the  First  workshop.  Cordell 
Green  of  Kestrel  Institute,  and  Dave  Gunning,  along  with  others  at  Kestrel  and  AL/HRG, 
reviewed  numerous  drafts  and  provided  valuable  feedback  along  the  way.  Thank  you  all. 


David  Zimmerman 
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SUMMARY 

This  report  describes  the  results  of  a  workshop  held  in  Palo  Alto,  California, 
(September  1991)  and  a  follow-up  session  held  at  Wright-Patterson  Air  Force  Base  (March 
1992)  to  explore  emerging  technologies  which  could  be  applied  to  the  automated  produc¬ 
tion  and  presentation  of  aircraft  maintenance  information  in  digital  form.  The  workshops 
were  sponsored  by  the  Logistics  Research  Division  of  Armstrong  Laboratory  to  help  estab¬ 
lish  a  research  direction  for  the  next  20  years  that  incorporates  concepts  being  developed  in 
university  laboratories  and  by  commercial  enterprises. 


iv 


CONTENTS 


Ease 


PREFACE . Hi 

SUMMARY . iv 

L  INTRODUCTION . 1 

Processes  and  Products . 1 

A  Glimpse  of  the  Future . 3 

Organization  of  the  Report . 6 

0.  FOUNDATIONS:  CURRENT  AND  NEXT  GENERATION . 7 

Current  Approach  (F-16) . 8 

Next  Generation  Approach  (F-22) . 8 

Design . 9 

Integrated  Logistics  Support . 9 

Technical  Authoring . 9 

Electronic  Presentation . 11 

in.  TECHNOLOGIES  FOR  THE  FUTURE . 13 

Product  Data . 13 

Qualitative  Physics . 14 

Automated  Planning . 14 

Human  Modeling . 15 

Automatic  Text  Generation . 15 

Graphics  Synthesis . 16 

Automated  Verification . 16 

Virtual  Reality . 17 

IV.  RESEARCH  AND  DEVELOPMENT  MILESTONES . 18 

Integrated  Product  Development . 18 

Product  Knowledge  Base . 19 

Engineering  Analysis . 20 

Task  Analysis . 22 

Text  Generation  and  Graphics  Synthesis . 23 

Procedural  Information . 25 

Descriptive  Information . 26 

Dynamic  Presentation  and  Virtual  Reality . 27 

Troubleshooting,  Replacement,  and  Repair . 27 

Capture  and  Review . 29 

V.  CONCLUSIONS  AND  REVIEW . 30 

Key  Themes . 30 

VI.  RESEARCH  RECOMMENDATIONS . 31 

Architectural  Design . 31 

Planning  Language  Foundation . 33 

Translators,  Critics,  and  Assistants . 34 

Initial  Agenda . 36 


v 


Ease 

REFERENCES . 40 

WORKSHOP  PARTICIPANTS . 43 

ACRONYMS . 47 

Figures 

Figure  Page 

1 .  A  Current  and  Future  View  of  Technical  Manual  Processes  and  Products . 2 

2 .  A  High-Level  View  of  Technical  Manual  Preparation . 7 

3 .  Maintenance  Information  Creation  —  Integrated  System  Concept . 10 

4.  Proposed  Functional  Architecture  and  Principal  Tools . 32 

5 .  Plan-Based  View  of  Automated  Technical  Manual  Creation . 33 

6 .  Integrated  Graphical  User  Interface  —  Author’s  Workstation  Concept . 39 


vi 


REPORT  OF  THE  WORKSHOPS:  AUTOMATED  GENERATION 
OF  ELECTRONIC  TECHNICAL  MANUALS 


I.  INTRODUCTION 

The  Logistics  Research  Division  of  Armstrong  Laboratory  (AL/HRG)  held  a  work¬ 
shop  in  Palo  Alto,  California,  (September  1991)  and  a  follow-up  session  at  Wright- 
Patterson  Air  Force  Base  (March  1992)  to  explore  emerging  technologies  which  could  be 
applied  to  the  automated  production  and  presentation  of  aircraft  maintenance  information 
through  Interactive  Electronic  Technical  Manuals  (IETMs)  .  The  participants  were 
primarily  from  universities  and  research  institutes,  but  also  included  representatives  from 
both  nonprofit  organizations  and  those  with  commercial  products  related  to  the  technical 
manual  production  process.  The  purpose  of  the  workshops  was  to  aid  AL/HRG  in 
establishing  a  research  direction  for  the  next  20  years  that  incorporates  concepts  being 
developed  in  university  laboratories  and  by  commercial  enterprises. 

Although  the  central  focus  of  the  workshop,  the  automated  generation  of  electronic 
manuals  is  only  one  phase  of  the  participants’  overall  vision  of  future  maintenance  infor¬ 
mation  support.  A  recurring  theme  of  the  workshop  was  that  this  future  must  involve  the 
cyclic  integration  of  supporting  technologies  across  a  wide  spectrum,  from  original  product 
engineering  design  and  data  management,  through  the  presentation  of  technical  instructions 
and  maintenance  information  to  end  users. 

Within  each  of  the  maintenance  information  production  phases  (identified  below),  de¬ 
velopmental  data  is  gathered  to  provide  feedback  to  earlier  phases.  The  potential  economies 
and  expected  leverage  of  such  integration  are  key  promises  in  the  fundamental  transition, 
now  under  way,  which  this  workshop  addressed.  Specifically,  the  workshops  addressed 
the  technological  developments  needed  to  move  from  paper  documents  largely  produced 
and  checked  for  accuracy  by  humans,  to  various  electronic  sources  produced,  verified,  and 
validated  largely  by  machine. 


Processes  and  Products 

The  development  of  maintenance  information  for  eventual  delivery  to  technicians  in¬ 
volves  a  series  of  interdependent  processes  and  products.  The  overview  in  Figure  1  illus¬ 
trates  the  key  elements  in  this  development,  their  current  form,  and  their  anticipated  form  in 
the  near-  and  long-term  future  (Gunning,  1991).  This  figure,  particularly  the  bottom  row, 
provides  a  rough  road  map  of  the  specific  technologies  considered  during  the  workshops. 

The  blocks  in  the  top  row  represent  processes;  the  words  above  and  to  the  right  of  the 
blocks  describe  the  output  of  the  processes.  The  second  row  shows  the  form  of  these 
products  within  the  current  development  environment:  Engineering  output  is  equipment 
design  via  paper  which,  in  turn,  serves  as  input  to  Logistics  Support  Analysis  (LSA).  The 
LSA  process  produces  LSA  records  and  tables,  also  via  paper,  which  a  technical  writer 
uses  to  describe  procedural  task  steps,  based  on  the  analysis  of  individual  maintenance 
tasks.  Task  analysis  is  a  key  issue  in  the  integration  of  LSA  with  technical  data.  It  is  also 
the  key  to  generating  maintenance  procedures  (more  or  less)  automatically.  Continuing  the 
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process,  an  illustrator  adds  drawings  to  the  text,  then  the  product  undergoes  Quality  Assur¬ 
ance  (QA)  through  visual  inspection.  Though  the  author  and  illustrator  each  store  their 
contributions  digitally,  as  Computer-Aided  Design  (CAD)  files,  the  final  validated  technical 
manual  is  delivered  to  the  technician  in  paper  form. 

The  third  row  represents  what  is  expected  in  the  year  2000:  Engineering  produces  a 
digital  file,  with  drawings  in  the  Interim  Graphics  Exchange  Specification  (IGES)  format. 
LSA  information  is  also  stored  digitally,  but  separate  from  the  engineering  data.  The  writer 
has  access  to  both  databases  and  produces  format-free  textual  instructions  in  accordance 
with  the  IETM  Data  Model.1  Illustrations,  as  digital  graphics,  will  be  linked  to  the  associ¬ 
ated  text  within  these  databases.  QA  is  performed  visually  via  a  simulation  screen,  similar 
to  the  technician’s  Portable  Maintenance  Aid  (PMA). 
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Figure  1 

A  Current  and  Future  View  of  Technical  Manual  Processes  and  Products 

The  next  row  shows  the  maintenance  information  products  as  envisioned  within 
roughly  10  years,  the  approximate  long-range  “look-ahead”  of  the  workshops.  The  text  at 
the  bottom  notes  several  of  the  particular  technologies  discussed,  such  as  Qualitive 
Physics,  Artificial  Intelligence  (Al)-planning,  three-dimensional  (3D)  graphics  synthesis 
and  Virtual  Reality  (VR),  and  automated  software  verification,  which  might  use  and  pro- 


1The  IETM  Data  Model  is  a  database  design  specification  for  organizing  and  cross-referencing  various 
maintenance  information  elements  (Fuller  etal.,  1991). 
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duce  these  products.  All  these  anticipated  technologies  and  their  interactions  are  described 
in  more  detail  in  later  sections  of  this  report 

Although  the  illustration  follows  the  linear  (serial)  way  in  which  individual  phases  are 
performed  today,  it  is  expected  that  the  overall  process  will  become  increasingly  “circular.” 
For  example,  logistics  analysis  results  will  feed  back  into  engineering  for  design  revisions, 
verification  technology  may  uncover  inconsistencies  which  will  trigger  regeneration  of 
textual  descriptions,  and  so  forth.  The  collective  effect  of  the  successful  implementation  of 
these  advanced  technologies  is  reflected  in  the  following  scenario,  presented  in  the  same 
spirit  as  the  “Sgt.  Bayshore”  scenario  of  some  13  years  ago.2 


A  Glimpse  of  the  Future 

Joe  Write  had  been  involved  in  creating  maintenance  information  for  20  years.  His 
first  job  as  a  technical  writer  was  on  the  F-16,  where  he  had  worked  for  6  years  before  be¬ 
ing  transferred  to  the  F-22  in  1996.  Prior  to  that,  he  had  worked  on  the  environmental 
control  system  of  the  F-16.  In  those  15  years  as  a  technical  writer,  he  had  seen  an 
evolution  from  paper-based  technical  manuals  to  computer-based  information  delivered  as 
neutral  data  and  displayed  on  a  screen.  However,  this  natural  though  dramatic  evolution 
did  not  prepare  him  for  the  last  5  years  and  the  even  more  dramatic  changes  he  had 
witnessed. 

The  changes  began  with  the  merger  of  several  companies  into  the  National  Airplane 
Company  (NAC),  prompted  by  a  need  to  survive  in  the  down-sized  world  of  defense 
spending,  and  they  continued  when  NAC  received  the  contract  for  a  successor  to  the  F-22, 
the  F-27.  The  winning  proposal  promised  to  implement,  within  an  integrated  production 
system,  results  of  work  sponsored  by  the  Department  of  Defense  (DoD)  and  its  component 
laboratories  since  the  initial  F-22  contracts.  The  specific  elements  to  be  implemented  and 
integrated  included: 

•  A  well-defined  specification  for  the  interpretation  and  exchange  of  product  data. 

•  A  mature  task  analysis  capability  including  human-modeling,  planning,  and  ani 
mation,  based  upon  detailed  knowledge  about  both  equipment  and  personnel. 

•  Generation  of  maintenance  information  through  automated  synthesis  of  text, 
graphics,  and  sound. 

•  Virtual  worlds  which  simulated  the  aircraft  and  overall  maintenance  environment 
in  a  realistic  3D  space. 

On  the  F-16,  Joe  had  worked  with  paper  source  data  and  interviews  with  engineers  to 
create  his  isolated  database  of  maintenance  information.  Although  much  of  the  design  had 
been  created  on  CAD  terminals,  he  could  not  access  it  directly.  On  the  F-22,  all  engineer- 


2Some  13  years  ago  a  short  scenario  depicting  a  maintenance  technician  named  Sgt.  Bayshore  was  written 
(Johnson,  1981).  It  centered  on  the  use  of  a  portable  computer,  to  provide  technicians  with  all  the 
information,  including  technical  data,  needed  to  do  their  jobs.  The  concept  grew  into  a  program  now  known 
as  the  Integrated  Maintenance  Information  System  (IMIS)  (Link  et  al.,  1987).  This  visionary  approach, 
divided  into  achievable  research  projects,  has  become  the  baseline  maintenance  concept  for  the  F-22. 
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ing  design  was  performed  on  workstations  linked  to  a  common  network;  thus,  all  this  de¬ 
sign  information  was  available  to  Joe.  LSA  data  was  created  similarly  and  was  also  di¬ 
rectly  available. 

However,  most  information  was  still  developed  serially;  engineering  finished  the  de¬ 
sign  before  LSA  was  accomplished,  and  LSA  had  to  be  completed  before  Joe  could  create 
his  information.  Validation  of  maintenance  information  had  also  changed  on  the  F-22  con¬ 
tract,  but  engineers  and  technical  reviewers  still  had  to  step  through  the  procedures  to  as¬ 
sure  their  accuracy. 

Joe  considered  the  F-27  requirements  to  be  an  order  of  magnitude  beyond  those  of  the 
F-22.  Design  engineers  no  longer  created  systems  in  isolation.  Every  component,  sub- 
assembly,  and  assembly  underwent  an  early,  automated  analysis  of  what  was  required  to 
build  and  repair  it.  If  the  analysis  uncovered  problems  with  the  design,  Al-based  software 
would  propose  a  solution.  Automated  human  or  machine  models  could  be  animated, 
graphically  enacting  the  manufacturing  and/or  maintenance  problems  discovered.  In  some 
instances,  the  designer  was  able  to  put  on  a  sensor  suit  and  helmet  and  enter  a  virtual  world 
which  simulated  that  of  the  technician. 

Joe  had  witnessed  the  evolution  of  logistic  support  analysis  from  a  single  discipline, 
overseeing  design  as  it  impacted  logistics  requirements,  into  a  collection  of  interrelated  pro¬ 
grams  and  databases  which  provided  more  immediate  and  much  more  rigorous  analyses. 
Interim  evaluations  accompanied  each  design  step  rather  than  being  performed  after  the 
fact.  One  major  benefit  Joe  observed  was  the  tremendous  reduction  in  paper  products, 
previously  necessitated  by  design  and  LSA  specifications,  as  well  as  contractual  delivery 
requirements.  These  changes  affected  his  job  in  several  ways. 

•  He  no  longer  directly  created  text;  it  was  generated  as  a  product  of  design  and 
computer-aided  task  analysis.  Using  standardized  style  libraries,  both  text-  and 
voice-based  procedural  descriptions  could  be  generated  to  address  a  variety  of 
technician  skill  levels  and  maintenance  contexts. 

•  No  illustrators  were  needed;  most  equipment  diagrams  were  generated  from  en 
gineering  data  and  CAD  graphics.  The  automatic  planning  programs  which  gen 
erated  the  procedures  also  extracted  or  synthesized  supporting  illustrations  from 
design  drawings  and  structural  specifications. 

•  Certification  of  procedures,  which  earlier  meant  validation  through  human 
“walk-throughs,”  was  largely  automated.  Specialized  software,  drawing  upon  a 
knowledge  base  of  consistency  requirements  and  presentation  rules,  now 
supplied  formal  validation  and  verification,  assuring  that  the  appropriate 
information  would  be  correctly  sequenced  and  displayed  during  any  potential 
maintenance  scenario. 

Though  Joe  reviewed  the  procedures  and  supporting  illustrations,  the  detailed  testing 
was  all  done  by  machine.  If  he  found  problems,  he  met  with  the  responsible  designers  and 
programmers.  Together  they  worked  out  solutions  and  changed  the  certification  system 
knowledge  bases  as  required. 

As  Joe  was  relieved  of  these  more  mundane  tasks,  he  was  given  a  new  one:  the  per¬ 
formance  of  maintenance  procedures  in  a  computer-generated  environment.  Donning  a 


4 


sensor-suit,  complete  with  gloves  and  helmet,  and  connected  to  a  simulation  computer,  he 
experienced  a  virtual  reality  in  which  he  removed  parts  according  to  instructions  and  re¬ 
paired  or  replaced  them  as  necessary. 

The  computer-generated  models  displayed  on  his  visor  appeared  real.  Tactile  sensa¬ 
tions  were  created  by  his  gloves;  sounds  of  the  flight  line  embellished  the  illusion.  If  a  task 
required  two  people,  another  acting  "technician"  also  put  on  a  suit  to  assist.  Both  Joe  and 
his  peer  were  in  the  same  “world,”  seeing  and  hearing  each  other  as  if  it  were  real. 

This  VR  world  also  allowed  Joe  to  act  as  an  “author.”  As  he  physically  performed 
tasks  on  the  virtual  aircraft  in  the  virtual  world,  the  computer  recorded  his  motions,  gen¬ 
erating  a  series  of  steps  which  constituted  the  procedure.  In  both  cases,  the  product  knowl¬ 
edge  base  supplied  the  engineering  detail  for  the  3D  equipment  renderings. 

Joe's  individual  responsibilities  were  not  the  only  logistics  elements  which  underwent 
dramatic  change  on  the  F-27. 

•  Staffing  of  spare  parts  personnel  was  reduced  to  a  fraction  of  earlier  needs,  since 
spares  were  automatically  identified  by  maintenance  levels  and  quantities. 

•  Training  became  part  of  the  Information  Monitoring  Group  of  which  Joe  was  a 
member.  Equipment  mockups  were  replaced  by  VR  models. 

Technician  training  still  consisted  of  both  diagnosing  problems  and  performing  mainte¬ 
nance  procedures,  but  training  media  was  again  generated  from  engineering  and  CAD 
sources.  Equipment  simulations  required  for  training  could  be  generated  in  a  virtual  world, 
eliminating  the  need  to  construct  physical  "mock-ups."  Virtual  overlays  and  “x-ray”  views 
supported  diagnosis  and  troubleshooting.  The  suspected  component(s)  of  a  given  sub- 
assembly  could  be  highlighted  with  colors  corresponding  to  their  fault  status;  internal  de¬ 
tails  could  be  “seen”  beneath  the  surface. 

Actual  maintenance  performance  was  similarly  supported  by  such  capabilities,  includ¬ 
ing  view  rotation,  interactive  zooming,  and  animation.  The  technician’s  PMA  screen  gave 
way  to  “heads-up”  displays  and  “see-through”  VR  goggles,  which  cou  j  show  exploded 
and  rotated  views  of  the  individual  parts  of  a  complex  (real)  component  as  he  looked  at  it. 
Descriptive  information,  such  as  operational  theory,  was  also  delivered  in  this  way.  Inter¬ 
nal  mechanics  and  fluid  flows  could  be  animated,  to  appear  above  the  (real)  assemblies  be¬ 
ing  examined. 

The  changes  Joe  and  his  co-workers  experienced  did  not  happen  overnight.  They 
were  based  on  incremental  milestones  developed  as  part  of  a  total  vision.  Individual  pro¬ 
jects  were  funded  at  various  university,  commercial,  and  defense  laboratories  and  orches¬ 
trated  by  a  central  office.  Interim  results  from  each  project  were  constantly  evaluated  for 
impact  on  other  projects,  and  appropriate  changes  made.  The  outcome  appears  to  be 
magic.  It  isn't.  It  is  only  the  application  and  extension  of  existing  capabilities  toward  a  vi¬ 
sionary  goal. 
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Organization  of  the  Report 

The  remainder  of  this  report  is  organized  into  five  sections.  Section  II  provides  some 
background  by  examining  the  foundations  of  the  technical  manual  production  process  as  it 
exists  now  on  the  F-16  and  as  it  will  be  implemented  on  the  F-22.  The  F-22  approach  is 
the  baseline  from  which  newer  concepts,  discussed  in  this  report,  will  be  developed. 
Those  already  familiar  with  IMIS  and  the  current  “state  of  the  art”  in  technical  manual  pro¬ 
duction  may  wish  to  skip  or  briefly  review  this  section. 

Section  III  introduces  the  applicable  technologies,  as  presented  during  the  first  work¬ 
shop.  Each  technology  area  is  defined  and  briefly  overviewed.  Section  IV  continues  pre¬ 
sentation  of  the  first  workshop’s  results  with  suggestions  for  short-,  medium-,  and  long¬ 
term  research  and  development  milestones.  The  suggested  milestones  are  collected  under 
three  principal  areas  for  advancement,  corresponding  to  the  three  basic  phases  of  mainte¬ 
nance  information  support  presently  identified.  These  sections  thus  extend  and  generalize 
current  concepts  and  activities  regarding:  (i)  acquisition  of  engineering  source  data, 
(ii)  technical  authoring,  and  (iii)  presentation  to  the  technician/user. 

Section  V  summarizes  the  results  of  the  first  workshop,  abstracting  several  key  themes 
which  emerged.  Section  VI  provides  an  initial  research  agenda.  The  specific  project  items 
of  the  agenda  were  derived  from  the  second  workshop,  in  which  a  subset  of  the  original 
participants  gathered  to  narrow  their  focus  and  agree  on  a  program  for  prospective  funding. 
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II.  FOUNDATIONS:  CURRENT  AND  NEXT  GENERATION 

Technical  manuals,  termed  technical  orders  (TOs)  by  the  Air  Force,  are  the  only  offi¬ 
cial  source  for  information  on  how  to  operate,  diagnose,  and  repair  equipment.  TOs  con¬ 
stitute  a  major  expenditure  during  the  operational  life  of  a  weapon  system.  The  costs  asso¬ 
ciated  with  TO  development,  maintenance,  updating,  distribution,  storage,  and  transporta¬ 
tion  have  increased  dramatically  for  several  reasons. 

•  Equipment  has  become  more  sophisticated,  particularly  with  the  introduction  of 
on-board  computers. 

•  Specifications  covering  the  content  and  format  of  the  manuals  have  evolved,  re 
quiring  more  detailed  coverage  directed  at  less  experienced  technicians. 

•  The  advancements  cited  above  have  led  to  a  requirement  for  more  pages  and 
more  manuals. 

Figure  2  shows  the  basic  steps  in  technical  manual  preparation.  Source  data  is  used  by 
the  author  to  generate  the  information,  which  is  then  delivered  to  the  user.  Creation  of  the 
manuals  involves  three  major  elements:  textual  instructions,  illustrations  to  support  the 
text,  and  the  merging  of  text  and  illustrations  to  produce  the  final  pages  of  the  TO. 


Figure  2 

A  High-Level  View  of  Technical  Manual  Preparation 


Historically,  TO  text  has  been  written  by  authors  having  expertise  in  a  specific  field 
(e.g.,  electronics,  hydraulics,  structures).  Design  and  manufacturing  information  served 
as  the  primary  source  data,  augmented  by  dialogues  with  engineers.  The  manuscript  was 
either  typed  or  handwritten. 

Illustrations  necessary  to  support  the  text  were  either  sketched  by  the  writer  and  given 
to  an  illustrator  or  verbally  described.  The  illustrator,  working  at  a  drafting  table,  prepared 
the  illustration  using  lettering  pens,  templates,  "rub-ons,"  and  other  artistic  devices. 

When  both  writer  and  illustrator  completed  their  work,  information  was  passed  to  a 
production  department.  Here  the  text  was  prepared  on  a  type-setting  device,  and  the  output 
was  pasted  onto  page  layout  sheets  and  integrated  with  the  illustrations  as  appropriate.  The 
front  matter  was  prepared  by  the  production  department  after  all  changes  required  by  the 
writer  and/or  quality  control  personnel  were  made.  Finally,  the  completed  pages  were  sent 
to  the  printer  where  the  deliverable  product,  printed  pages  or  negatives,  was  prepared. 
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Current  Approach  (F-16) 

With  the  advent  of  the  F-16,  increasing  complexity,  both  in  the  equipment  and  level  of 
detail  in  TO  specifications,  forced  contractors  to  find  ways  to  produce  the  increasing  num¬ 
ber  of  pages  in  a  timely  manner  and  at  an  acceptable  cost.  This  was  accomplished  by 
(separately)  applying  computer  technology  to  each  step  in  the  process. 

•  Writers  were  given  computers  or  computer  terminals,  where  their  text  was  cap 
tured  and  used  by  the  production  personnel.  Handwritten  or  typed  manuscripts 
virtually  disappeared. 

•  Dlustrators  were  given  either  CAD  or  specialized  illustration  terminals  on  which 
to  prepare  the  artwork. 

•  Production  personnel  were  given  page-layout  terminals  on  which  the  text  and  il 
lustrations  could  be  merged  within  a  template  which  complied  with  the  particular 
specifications  governing  the  TO. 

Though  successful,  this  sort  of  “office  automation  technology”  failed  to  produce  the 
expected  gains  in  productivity  used  to  justify  its  implementation  costs.  The  cognitive  pro¬ 
cess  preceding  the  actual  text  creation,  illustration  sketch,  or  page  preparation  was  still 
largely  unsupported. 

A  major  fault  with  the  F-16  approach  was  "automating"  the  three  processes  indepen¬ 
dently.  Authors  with  character  terminals  could  not  access  the  illustration  for  an  electronic 
review.  Illustrators  could  not  access  the  text  which  their  drawing  supported;  they  could  not 
see  it  in  context.  Neither  group  could  see  the  final  page  as  it  was  laid  out  by  the  production 
staff. 

Another  major  fault  was  that  the  technical  manual  department  was  "automated"  inde¬ 
pendent  of  all  other  disciplines — engineering,  manufacturing,  etc.  Department  personnel 
still  used  paper  products  as  their  source  data,  manually  converting  it  to  digital  form  in  their 
own  machines,  and  thus  creating  a  largely  redundant  database. 

The  new  specifications  meant  that  an  F-16  technician  (the  end  user  of  the  products)  re¬ 
ceived  hundreds  of  technical  manuals  containing  tens  of  thousands  of  pages.  Although 
these  did  contain  the  information  he  needed  to  do  his  work,  the  information  was  frequently 
hard  to  find,  thus  increasing  the  man-hours  required  to  repair  an  aircraft. 

The  cost  of  a  TO  library,  with  attendant  personnel,  also  increased.  The  numerous  TOs 
required  more  storage  area  and  were  repeatedly  updated  with  hundreds  of  changes.  A  sub¬ 
set  of  the  library  had  to  be  packed  for  field  deployment,  occupying  much  of  the  volume  and 
weight  required  to  support  the  move. 


Next  Generation  Approach  (F-22) 

With  the  next  generation  aircraft,  maintenance  information  processes  will  become  more 
integrated  and  less  redundant.  On  the  F-22,  designers  and  authors  will  be  able  to  access 
technical  databases  and  use  the  (source  data)  information  created  upstream  from  them.  The 
following  paragraphs  describe  several  facets  of  the  planned  approach. 
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Design 


Design  of  a  weapon  system  (aircraft  and/or  support  equipment)  will  be  done  exclu¬ 
sively  on  computer  terminals.  A  database  containing  all  drawings  for  manufacturing  will 
be  generated,  with  bills  of  material,  parts  lists,  etc.,  available  as  ASCII  files.  Most  of  the 
design  illustrations  will  be  3D,  based  on  underlying  solid  models. 


Integrated  Logistics  Support 

Integrated  Logistics  Support  (ILS)  will  operate  as  an  integral  pan  of  design  engineer¬ 
ing.  It  will  identify  ways  in  which  design  changes  can  improve  both  supportability  and  the 
resources  needed  to  support  a  fielded  system  (Jones,  1991).  ILS  disciplines  include: 

•  Maintenance  Planning 

•  Supply  Support 

•  Technical  Data 

•  Facilities 

•  Manpower  and  Personnel 

•  Training  and  Training  Support 

•  Support  Equipment 

•  Computer  Resources  Support 

•  Packaging,  Handling,  Storage,  and  Transportability 

•  Design  Interface 

The  findings  of  the  ILS  disciplines  will  be  recorded  in  a  database  which  can  produce 
reports  or  electronic  views  of  the  data.  This  is  the  idea  of  the  Design  Evaluation  for  Per¬ 
sonnel,  Training,  and  Equipment  (DEPTH)  program  (Boyle,  1991).  The  database  will  in¬ 
clude  identification  of  maintenance  tasks,  steps  to  perform  those  tasks,  personnel  require¬ 
ments,  expected  completion  time,  required  equipment,  and  other  pertinent  information,  all 
of  which  will  be  directly  usable  in  the  maintenance  instructions. 


Technical  Authoring 

Technical  manuals  will  be  created  on  high-resolution  terminals  with  graphics  capabili¬ 
ties.  The  writer  will  be  able  to  access  the  ILS  and  design  databases  via  windows,  and  copy 
information  needed  to  complete  a  task.  This  is  a  foundation  of  the  Integrated  Maintenance 
Information  System  (IMIS)  concept  (Link  et  al.,  1987).  The  writers'  database  will  have 
aspects  of  a  hypertext-like  architecture  but  will  be  custom-designed  to  comply  with  the 
delivery  requirements  of  a  format-neutral  database. 
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Figure  3 

Maintenance  Information  Creation — Integrated  System  Concept 

Figure  3  illustrates  the  integrated  IMIS  authoring  system  in  which  the  writer  (and  illus¬ 
trator)  will  be  supported  not  only  by  a  sophisticated  word  processor  but  by  programs  such 
as  the  following. 

•  Graphics  Editor:  A  primitive  graphics  capability  (compared  to  the  illustrator’s) 
which  allows  engineering  drawings  to  be  accessed,  copied,  and  changed  as  re 
quired.  Rudimentary  sketches  can  also  be  created  for  submission  to  illustrators. 


10 


•  Standard  Word  Lists:  Programs  which  check  that  procedural  instructions  are 
consistent  and  use  only  approved  and  standardized  terminology  and  action  verbs. 

•  Reading  Level  Support:  Programs  to  check  word  and  sentence  length  to  assure 
that  the  reading  grade  level  of  the  information  does  not  exceed  that  of  the  target 
user.  Spelling  checks  and  on-line  correction  will  also  be  available. 

•  Run-Time  Emulation:  Emulation  capabilities  will  allow  the  writer  to  view  a  pre 
sentation  of  the  authored  text/graphics,  just  as  a  technician  would  view  it  during 
a  maintenance  scenario.  This  helps  assure  the  writer  that  the  information  has 
been  created  properly,  linked  with  appropriate  graphics,  contains  cautionary 
information  in  the  proper  places,  and  so  forth,  as  required  by  presentation 
specifications. 

The  illustrator  will  have  access  to  writer-prepared  sketches,  design  files,  and  a  library 
of  previously  prepared  illustrations.  Through  windowing  he  will  be  able  to  copy  any  file, 
and  merge  and  make  changes  as  required  to  support  the  overall  technical  presentation. 

The  writer  and  the  illustrator  will  each  have  in-process  files,  to  which  they  alone  have 
access.  When  they  have  completed  their  tasks,  the  work  is  moved  into  another  database 
where  it  is  subjected  to  quality  assurance  and  validation  requirements  as  a  completed  docu¬ 
ment.  When  these  processes  are  finished  the  merged  file  is  delivered  to  the  customer/user 
and  archived.  The  writer's  and  illustrator's  files  are  also  archived,  independently  of  the 
merged  file(s),  for  future  reference,  copying,  or  modification. 


Electronic  Presentation 

The  culmination  of  these  efforts  is  delivery  of  information,  describing  how  to  perform 
maintenance  actions  on  a  specific  system,  to  a  given  maintenance  technician.  On  the  F-22, 
information  will  be  delivered  via  the  electronic  display  of  a  PMA.  The  information  will  be 
selected  and  filtered  for  his  specific  system  and  its  particular  configuration;  he  will  not  have 
to  sift  through  screen  displays  which  do  not  apply  to  his  current  situation.  He  will  be  given 
information  appropriate  to  his  level  of  training  and  experience,  eliminating  the  paper-based 
problem  of  targeting  the  presentation  to  the  lowest  skill-level  technician. 

The  availability  of  maintenance  history,  similar  to  that  which  a  doctor  has  regarding  a 
patient's  medical  history,  is  another  important  feature.  This  will  allow  the  technician  to  see 
what  has  previously  been  done  to  his  "patient"  to  solve  the  same  or  similar  problems.  If  a 
technician  encounters  difficulties  with  some  segment  of  the  presentation,  he  can  electroni¬ 
cally  "mark"  it  so  that  it  can  be  analyzed  by  the  contractor  for  accuracy.  If  he  is  certain 
about  the  error  he  can  create  a  "note"  or  "bookmark"  to  which  he  will  automatically  be 
alerted  the  next  time  he  runs  the  procedure,  if  it  has  not  been  officially  corrected. 

The  PMA  will  maintain  an  audit  trail  of  the  technician's  activities:  the  path  he  followed 
through  the  information,  how  much  time  he  devoted  to  particular  screen  displays,  and  what 
maintenance  actions  he  performed.  This  information,  along  with  that  of  other  technicians, 
will  be  up-loaded  into  a  central  database  where  it  can  be  analyzed  to  determine  possible  de¬ 
ficiencies  in  either  the  information  supplied  or  the  technician’s  effectiveness.  If  a  signifi¬ 
cant  number  of  technicians  experience  the  same  types  of  problems,  the  information  may  be 
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re-analyzed  for  understandability.  If  a  particular  technician  establishes  a  pattern  of  not  un¬ 
derstanding  the  presentations,  additional  training  may  be  recommended. 

These  are  all  substantive  improvements  to  the  process  of  technical  information  creation 
and  delivery,  and  will  increase  both  production  efficiency  and  presentation  quality.  How¬ 
ever,  they  are  by  no  means  all  that  may  be  envisioned  and  planned  for.  A  number  of 
emerging  technologies,  if  properly  developed  and  applied,  could  support  even  greater 
changes,  such  as  (a)  complete,  standardized  engineering  and  logistics  information; 

(b)  maintained  and  updated  information  throughout  a  product’s  service  life;  and 

(c)  automated  generation  and  correctness  certification  of  sophisticated  multimedia 
presentations.  These  benefits  could  be  derived  through  the  use  of  sensory  enhancements 
verging  on  the  dream-like — mechanical  animations,  dynamic  overlays,  holograms,  and 
virtual  reality. 

Building  upon  the  foundations  described  above,  the  first  workshop  identified  a  num¬ 
ber  of  research  directions  essential  to  maintenance  information  support  in  the  generation  to 
follow.  By  extending  and  applying  current  work  in  these  areas,  we  can  take  the  first  steps 
toward  Joe  Write’s  world. 
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III.  TECHNOLOGIES  FOR  THE  FUTURE 


Future  maintenance  information  support,  as  envisioned  for  the  year  2000  and  beyond, 
depends  on  the  integration  of  a  range  of  individual  technologies  to  extend  the  baselines  de¬ 
scribed  above.  Accordingly,  the  workshops  brought  together  a  number  of  leading  experts 
in  some  of  the  emerging  technologies  thought  to  be  most  crucial.3  In  addition  to  the  pre¬ 
sentations  of  current  work  in  each  specific  area,  several  group  discussions  during  the  first 
workshop  afforded  participants  the  opportunity  to  place  their  results  within  the  overall 
context  of  maintenance  information  and  electronic  technical  manuals  and  consider  how  their 
techniques  might  be  composed  with  other  advances. 

This  section  overviews  each  area  of  the  technology  addressed  and  where  it  fits  within 
the  overall  development  process.  Several  of  the  technologies  overlap,  some  have  potential 
application  in  more  than  one  phase,  and  others  apply  across  the  production  cycle  (e.g.,  in¬ 
tegrating  or  quality  assurance  technologies  not  closely  associated  with  any  specific  process 
or  product).  For  instance,  software  consistency  checking  and  certification  techniques  are 
applicable  to  all  phases  of  electronic  TO  production. 

The  technology  areas  specifically  addressed  during  the  workshop  were: 

•  Product  Data 

•  Qualitative  Physics 

•  Automated  Planning 

•  Human  Modeling 

•  Automatic  Text  Generation 

•  Graphics  Synthesis 

•  Automated  Verification 

•  Virtual  Reality 

The  workshop  also  reviewed  existing  processes  and  procedures  for  ILS/LSA,  includ¬ 
ing  management  of  technical  data.  The  following  paragraphs  give  a  short  description  of 
each  area;  more  detailed  discussions  may  be  found  in  the  papers  listed  in  the  references. 


Product  Data 

Product  data,  in  its  fullest  sense,  denotes  all  the  data  elements  which  define  a  product 
for  all  applications  over  its  expected  life  cycle,  where  life  cycle  includes  design,  analysis, 
test,  inspection,  and  product  support.  The  data  elements  required  include  geometry,  toler¬ 
ances,  material  properties,  surface  finishes,  logistics  support  information,  and  other  at¬ 
tributes  and  features  that  completely  define  a  component  part  or  assembly. 


3  A  list  of  the  participants,  and  a  brief  biography  of  each,  is  presented  in  the  appendix  . 
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A  major  effort  is  under  way  to  define  product  data  in  a  specification  entitled  Product 
Data  Exchange  using  STEP  (PDES),  where  STEP  is  the  Standard  for  The  Exchange  of 
Product  data  (Wilson,  1991).  PDES  is  intended  to  result  in  a  national  standard  compatible 
with  STEP.  The  primary  objective  of  PDES  is  the  development  of  a  standardized,  neutral, 
computer-based  definition  of  product  data  which  supports  the  “seamless,”  integrated  use  of 
product  data  for  different  applications. 

Product  information  consists  of  not  only  data,  but  the  relationships  between  data  and 
the  meanings  attached  to  it.  The  fundamental  concept  of  PDES  is  the  standardized  repre¬ 
sentation  of  (directly)  computer-intelligible  information  and  interrelationships,  versus  data 
which  becomes  information  only  because  it  is  human-intelligible.  PDES  is  directed  at  pro¬ 
viding  both  the  rationale  and  formal  foundation  for  information  extraction  from  product 
data. 


Qualitative  Physics 

Qualitative  physics  addresses  reasoning  about  the  physical  world,  with  the  goal  of 
capturing  both  the  “common  sense”  mechanical  intuition  of  humans  and  the  quantitative  de¬ 
scriptions  of  classical  physics  (Forbus,  1988).  The  hope  and  intent  is  to  achieve  a  degree 
of  systematic  coverage  and  uniformity  far  in  excess  of  today's  knowledge-based  systems. 
Expert  systems  today  contain  encoded  knowledge  about  a  particular  domain  for  a  particular 
purpose,  while  qualitative  physics  strives  to  create  wide-coverage,  multipurpose  domain 
systems. 

Current  concerns  in  qualitative  physics  include  dynamics  (what  causes  systems  to 
change  over  time),  kinematics  (the  spatial  reasoning  of  mechanical  common  sense),  and 
reasoning  styles  (ways  to  exploit  the  knowledge  in  such  areas  as  qualitative  simulation, 
envisioning,  recognition,  and  measurement  interpretation)  (deKleer,  1989). 

Potential  applications  for  qualitative  physics  exist  in  automated  planning,  particularly 
in  the  generation  of  procedures.  The  design  of  new  hardware  needs  to  incorporate  the  in¬ 
formation  necessary  to  generate  maintenance  procedures  automatically,  develop  procedures 
for  operating  it,  diagnose  its  failures,  and  repair  it.  As  computers  are  enlisted  to  generate 
complex  hardware  designs,  they  can  also  begin  to  generate  the  supporting  information. 


Automated  Planning 

Automated  planning  concerns  the  generation  of  a  plan,  or  composition  of  procedural 
steps  toward  a  goal,  which  is  one  possible  solution  to  a  particular  problem.  The  individual 
steps  are  typically  operator  schemata ,  each  of  which  characterizes  a  class  of  possible  ac¬ 
tions  (Hendler  et  al.,  1990).  Instantiation  of  the  variables  defined  by  an  action  class  pro¬ 
duces  a  specific  action  for  a  given  situation,  according  to  the  substitution  values  the  situa¬ 
tion  implies. 

In  general,  the  input  to  an  Al-planning  system  is  a  set  of  operator  schemata  appropriate 
to  the  application  domain  and  a  problem ,  characterized  by  an  initial-state  description  and  a 
goal.  The  initial  state  describes  the  planners  “world”  as  it  is,  and  the  goal  describes  how  it 
should  be  after  execution  of  the  plan.  The  goal  is  often  made  up  of  several  subgoals 
(Sacerdoti,  1975). 
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A  plan  is  a  solution  to  the  problem  if  the  plan  is  applicable  to  the  initial  state  and  after 
plan  execution,  the  goal  is  true.  A  plan  (or  subplan)  is  applicable  if  all  the  preconditions  for 
the  execution  of  its  first  operator  hold  in  the  current  state.  Repeated  analysis  of  the  required 
preconditions,  across  the  composed  effect  of  applied  operators,  (termed  “temporal  projec¬ 
tion”)  can  determine  whether  all  operators  may  be  applied  in  order,  starting  from  a  given 
initial  state. 


Human  Modeling 

Human  models  are  simplified  representations  of  human  figures  implemented  with 
computer-graphics  tools.  Such  models  are  often  used  with  CAD  systems  to  help  evaluate 
the  ergonomic  qualities  of  potential  human/machine  designs.  Physical  human  factors  such 
as  body  fit,  reach,  strength,  and  movement  can  be  evaluated  with  these  models  before 
equipment  fabrication  begins. 

Advances  in  computer  technology  will  allow  a  much  wider  array  of  human  factors  to 
be  captured  in  the  near  future.  The  best  features  of  the  best-known  human  models.  Jack 
(Badler,  1991)  and  Crew  Chief  (Easterly  &  Ianni,  1991),  are  to  be  combined  in  a  “super” 
human  modeling  system  under  the  DEPTH  program  (Boyle  et  al.,  1990).  Animated  mod¬ 
els  controlled  by  natural  language  “scripting”  should  provide  a  powerful  visual  adjunct  to 
automatic  generation  and  validation  of  maintenance  procedures. 

Al-planning  tools  are  central  to  both  the  animation  of  human  forms  and  the  generation 
of  procedural  descriptions  for  maintenance  tasks;  indeed,  the  specific  tools  required  may  be 
identical.  In  the  future,  human-model  animations  might  be  used  directly  in  maintenance  il¬ 
lustrations  and  will  provide  a  “reality  check”  for  automatically  generated  procedural  de¬ 
scriptions.  The  DEPTH  program  intends  to  approach  this  from  a  different  perspective.  It 
is  possible  that  the  planning  models  produced  by  animation  technology  can  provide  the  ba¬ 
sis  for  generating  procedural  descriptions. 


Automatic  Text  Generation 

The  goal  of  automatic  text  generation  is  (ultimately)  to  eliminate  dependence  on  human 
authors  for  maintenance  task  descriptions.  Instead,  natural  language  instructions,  tailored 
to  the  situation  at  hand,  will  be  mechanically  generated  to  explain  each  pan  of  the  mainte¬ 
nance  task.  This  is  a  key  step  towards  decreasing  the  production  costs  and  increasing  the 
flexibility  and  customization  of  maintenance  information  delivered  as  text. 

As  presented  at  the  workshop,  automatic  text  generation  is  closely  related  to  automated 
planning.  In  this  case,  a  plan  is  constructed  for  the  information  content  of  an  explanation, 
detailing  the  kinds  of  information  to  include,  and  their  sequencing  for  the  particular  purpose 
(i.e.,  explanation  “goal”)  involved.  The  plan  schema  is  represented  as  a  graph,  which  can 
be  traversed  to  select  particular  content  elements  from  a  knowledge  base  for  the  object  be¬ 
ing  repaired.  A  text  generator  then  produces  English  sentences  from  the  hierarchy  of  con¬ 
tent  elements. 

Natural  language  generation  involves  the  interaction  of  both  syntactic  and  semantic 
constraints,  such  as  the  “voice”  of  the  sentence  to  be  produced  (e.g.,  active  or  passive), 
agreement  between  subject  and  verb,  the  context  established  by  previously  generated  sen- 
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tences,  and  so  forth.  For  equipment  maintenance,  standard  word  lists  of  preferred  verbs, 
nouns,  and  modifiers  should  simplify  the  word  selection  problem.  Within  a  given  display 
(screen),  selection  of  supporting  graphics  may  introduce  constraints  on  text  generation  as 
well. 


Graphics  Synthesis 

As  with  text  generation,  graphics  synthesis  aims  to  replace  human  illustrators  with 
software  modules  that  will  produce  appropriate  equipment  diagrams,  wiring  schematics, 
and  so  forth,  to  supplement  the  text.  A  content  plan  can  again  be  constructed  to  specify  the 
communicative  goals  of  a  requested  illustration.  A  knowledge  base  of  geometric  shapes, 
organized  with  respect  to  the  hierarchy  of  equipment  parts,  provides  the  basic  illustrations 
for  mechanical  assemblies  and  subassemblies.  A  rule-based  component  controls  the  com¬ 
position  and  evaluation  of  possible  illustrative  styles  regarding  viewpoint  and  perspective, 
shading  and  highlighting,  use  of  insets  or  “cutaways,”  and  the  like.  The  final  illustration  is 
achieved  through  a  generate- and- test  approach,  with  backtracking  to  earlier  (partial) 
achievement  methods  when  incompatibilities  arise  (Seligmann  &  Feiner,  1989). 

Another  rule-based  approach  demonstrated  at  the  workshop  involves  successively 
transforming  expressions  in  a  data  description  language,  through  corresponding  expres¬ 
sions  in  a  visual  description  language,  and,  finally,  rendering  them  on  the  display  screen 
(Westfold  et  al.,  1990).  The  data  description  language  expresses  unformatted  relational  in¬ 
formation,  such  as  might  be  found  in  product  databases,  and  the  visual  description  lan¬ 
guage  includes  region  primitives  such  as  shape,  color,  and  position  as  well  as  various  con¬ 
structors  and  combining  forms.  Flexibility  in  the  individual  choices  of  visual  representa¬ 
tion  for  data  relation  allows,  for  example,  synthesis  of  either  (or  both)  a  table  or  “box  and 
arrow”  diagram  for  some  relational  expressions. 

In  general,  rule-based  approaches  permit  the  encoding  of  human  factors  constraints  as 
well  as  specific  display  hardware  limitations  or  strengths.  Such  elements  can  change  over 
time  or  for  different  maintenance  situations. 


Automated  Verification 

The  Air  Force  originally  encountered  software  testing  issues  in  developing  Operational 
Flight  Programs  (OFPs),  which  run  onboard  the  aircraft  and  have  historically  been  con¬ 
cerned  with  aspects  of  navigation,  weapons  delivery,  flight  control,  and  similar  aircraft  op¬ 
erations.  Due  to  flight  safety  concerns,  rigorous  specifications  and  procedures  have 
evolved  to  assure  the  accuracy  of  such  programs  before  they  are  loaded  onto  actual  aircraft. 
However,  since  testing  of  OFPs  is  primarily  done  on  aircraft  simulators  (by  humans),  the 
resulting  quality  assurances  are  largely  subjective. 

The  term  automated  verification  is  used  to  distinguish  formal  rule-  and  logic-based  in¬ 
ference  methods  performed  by  machine  from  the  informal  reviews  (traditionally  termed 
validation  and  verification )  performed  by  human  contractors  and  customers.  Automated 
verification  systems  typically  attempt  to  generate  a  formal  proof  that  the  product  (generally 
software),  in  some  well-defined  sense,  satisfies  a  given  formal  specification. 
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Though  such  mechanical  certification  has  been  notoriously  difficult  for  software  of  any 
appreciable  size  and  complexity,  the  task  can  be  made  more  tractable  if  behavior  is  re¬ 
stricted  to  forms  with  appropriate  foundations  for  mechanical  logic  (Wang  &  Goldberg, 
1991).  Similarly,  validation  (establishing  the  internal  consistency)  of,  for  example,  a 
database  of  maintenance  information,  can  be  feasible  if  the  structure  of  the  database  and 
desired  characteristics  of  the  stored  information  are  amenable  to  description  through  formal¬ 
ized  rules. 


Virtual  Reality 

Virtual  Reality  (VR)  is  an  artificial  environment  perceived  by  a  person  wearing  stereo 
eyepieces,  earphones,  and  (perhaps)  a  fully  sensored  suit  of  clothing,  all  of  which  are  con¬ 
nected  to  computers  and  act  as  peripherals.  The  computer  generates  sights  and  sounds 
(i.e.,  a  sensory  environment)  which  seems  virtually  “real”  to  the  user. 

The  idea  is  to  view  a  human  as  a  biological  mechanism  with  inputs  and  outputs — 
sight,  hearing,  touch — and  to  develop  a  strategy  for  providing  consistent  stimuli  which 
cover  these  sense  organs.  When  successfully  implemented,  the  person  has  the  sensation  of 
being  in  the  computer-generated  world,  experiencing  peripheral  vision,  real-life  sound,  ami 
tactile  sensations.  The  environments  created  today  are  mostly  cartoon-like,  lacking  the  de¬ 
tails  of  the  real  world,  but  are  becoming  less  so  every  year.  Texture  and  radiosity  (selective 
lighting)  have  recently  been  added,  and  detail  can  be  increased  as  required,  based  on  avail¬ 
able  computing  power. 

With  respect  to  maintenance  performance,  the  use  of  “see-through”  VR  goggles  (or 
possibly  “heads-up”  display  visors)  is  envisioned,  rather  than  enveloping  the  technician  in 
the  virtual  world.  Such  devices  could  provide  VR  images  showing  internal  mechanical  de¬ 
tails,  subassembly  explosions  and/or  rotations,  or  operation  animations,  superimposed  on 
(or  above)  the  real-world  components  upon  which  the  user  focuses  his  gaze.  The  identifi¬ 
cation  of  such  component  foci  might  be  done  manually  by  user  input  or  automatically  via 
miniature  video  input  devices  (attached  to  the  goggles)  and  associated  image  recognition 
software. 

Although  sometimes  viewed  as  a  futuristic  promise,  VR  is  already  both  a  government 
and  commercial  success.  It  is  being  used  by  the  National  Aeronautics  and  Space  Adminis¬ 
tration  (NASA)  to  simulate  planetary  exploration  and  by  the  military  to  train  pilots,  tank 
commanders,  and  soldiers.  It  has  commercial  applications  in  medicine,  architecture,  and 
education,  to  name  a  few.  Even  in  its  infancy,  VR  has  significant  potential  for  application 
in  the  world  of  aircraft  maintenance.  The  potential  is  limited  by  only  two  things:  the  vision 
for  its  application  and  the  computing  power  to  implement  that  vision. 


Maintenance  diagnostics  (i.e.,  fault  isolation),  a  large  topic  in  its  own  right,  was  not 
specifically  addressed  as  part  of  the  workshops;  however,  it  did  arise  in  panel  discussions 
because  of  its  obvious  importance  in  maintenance  procedures.  Product  design  was  also  not 
directly  addressed  but,  similarly,  came  under  consideration  with  respect  to  its  interaction 
with  logistics  analysis. 
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IV.  RESEARCH  AND  DEVELOPMENT  MILESTONES 


To  attain  the  maintenance  information  support  “vision”  described  in  the  first  section 
(Joe  Write’s  world),  advances  must  be  made  in  a  number  of  specific  areas.  This  section 
presents  a  collection  of  research  and  development  milestones,  organized  within  three  prin¬ 
cipal  categories,  which  may  be  regarded  as  research  thrusts  for  the  future.  The  approxi¬ 
mate  time  frame  for  these  milestones  is: 

•  Short  Term:  1  to  3  years. 

•  Medium  Term:  3  to  5  years. 

•  Long  Term:  5  to  7  years,  subject  to  availability  of  underlying  technology. 

Within  each  category,  the  corresponding  subsections  below  first  overview  the  broad 
area  addressed,  then  outline  potential  short-,  medium-,  and  long-range  research  goals 
within  particular  subareas.  While  it  is  recognized  that  the  necessary  integration  and  infor¬ 
mation  feedback  also  tend  to  blur  the  separation  of  individual  phases,  the  subheadings 
roughly  partition  the  field  along  the  classic  divisions  between  engineering  data  acquisition, 
technical  authoring,  and  delivery  to  the  technician.  They  suggest  the  extrapolation  of  these 
fundamental  processes  in  the  future. 


Integrated  Product  Development 

The  basic  activity  of  acquiring  and  analyzing  engineering  source  data  (in  order  to  de¬ 
velop  maintenance  procedures)  has  been  the  province  of  LSA.  LSA  refers  to  an  integrated 
analytic  process,  with  four  specific  goals: 

•  Influence  design  engineering, 

•  Identify  problems  and  cost  drivers, 

•  Develop  support  resource  requirements,  and 

•  Develop  logistics  support  database 

In  accordance  with  the  future  vision  of  increased  integration  and  feedback  between  lo¬ 
gistics  analyses  and  design  iterations,  this  report  expands  the  general  discipline  of  engineer¬ 
ing  data  acquisition  to  “Integrated  Product  Development,”  emphasizing  (as  in  Concurrent 
Engineering)  the  envisioned  close  connections  between  engineering  design  and  various 
forms  of  product  analysis  and  evaluation,  including  LSA  for  maintenance  information. 

There  was  general  agreement  among  workshop  participants  that  significant  advances  in 
this  area  will  depend  and  draw  upon  some  form  of  integrated  and  comprehensive  Product 
Knowledge  Base  (PKB).  More  than  just  a  common-access  database,  the  PKB  provides 
persistent  and  standardized  information  models  used  concurrently  by  product  engineer¬ 
ing/design  aiid  support  analyses,  as  well  as  supporting  Computer-Aided  Manufacturing 
(CAM). 
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The  PDES/STEP  endeavor,  which  aims  to  standardize  information  modeling  and  ex¬ 
change  throughout  the  entire  product  life  cycle,  may  be  viewed  as  moving  towards  such  a 
PKB.  To  adequately  support  the  emerging  analysis  technologies  discussed  in  this  work¬ 
shop  (e.g.,  qualitative  ohysics,  computerized  human-modeling,  and  Al-based  planning), 
their  information  requirements  and  products  must  be  taken  into  account  during  the  develop¬ 
ment  of  PKB  standards  f*>r  the  future. 

The  following  subsections  collect  potential  short-,  medium-,  and  long-range  goals 
within  product  development  and  LSA.  First  the  PKB  is  considered,  then  the  complemen¬ 
tary  topics  of  engineering  analysis  and  task  analysis. 


Product  Knowledge  Base 

The  overall  goal  of  the  PKB  is  to  provide  a  rigorous  formal  foundation  for  instantia¬ 
tion  and  exchange  of  product  information.  It  must  be  complete  and  comprehensive,  en¬ 
compassing  such  “known”  elements  as  geometric  solids,  mechanical  and  electrical  models, 
maintenance  task  simulations,  and  so  forth,  as  well  as  new  technologies  when  they  arise. 
Information  will  be  represented  in  a  neutral  internal  form  for  communication  between  a  va¬ 
riety  of  diverse  systems,  and  will  presumably  be  based  upon  one  or  more  conceptual 
schemas,  independent  of  any  particular  implementation  environment(s). 

Short  Term 

•  Maintenance  Requirements  Definition 

Outline  some  initial  problems  and  issues  from  the  maintenance  domain  to  provide  a 
focus  for  preliminary  research  work.  Select  an  appropriate  area  for  concentration, 
such  as  qualitative  physics  to  support  diagnosis  and  repair. 

•  Data  Representation  and  Translation 

Investigate  the  potential  for  the  intertranslation  of  engineering  source  data  between 
existing  product  description  languages  and  databases  (such  as  Express  (Wilson, 
1991)).  Determine  if  this  is  a  practical  long-range  approach;  apply  what  is  learned  to 
begin  development  of  standard  conceptual  schemas. 

•  Formalisms  for  Decision  Support 

Study  the  suitability  of  various  existing  formalisms  for  encoding  a  select  set  of 
maintenance  tasks.  Focus  on  planning  and  decision  issues  including  uncertain  action 
outcomes,  inspection  tasks,  and  task  hierarchies.  Investigate  possibilities  for  ac¬ 
commodating  these  notions  within  a  uniform  PKB. 
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Medium  Term 


•  Subsystem  Model  Development 

Develop  isolated  but  complete  logistics-oriented  models  of  several  aircraft  compo¬ 
nents  and/or  subsystems.  Use  these  small  examples  to  better  understand  what  the 
modeling  requirements  are  and  how  to  scale  up  to  larger  systems. 

•  Engineering  Change  Analysis 

Establish  dependency  links  between  CAD  models  and  graphics  (for  example). 
Study  how  a  decision-support  tool  might  be  used  to  perform  change  analysis.  In¬ 
vestigate  the  use  of  dependency  records  associated  with  planning  to  track  how 
changed  design  decisions  affect  other  aspects  of  a  maintenance  task. 

•  Knowledge  Base  Consistency  Monitoring 

Develop  some  simple  semantic  connections  between  different  representation  schemes 
within  the  PKB.  Investigate  the  application  of  automated  validation  techniques  to 
search  for  and  identify  potential  inconsistencies. 

•  Information  Access  -  Analyzer’s  Apprentice 

Develop  graphic  interfaces  and  simulation  facilities  to  access  information  from  the 
database  and  answer  simple  “what  if’  questions.  Build  a  prototype  learning  compo¬ 
nent  within  the  decision-support  system  to  act  as  an  advisor/apprentice. 

Long  Term 

•  Comprehensive  System  Modeling 

Develop  several  complete,  integrated,  hierarchical  system  models.  Abstract  an  ap¬ 
propriate  model  for  a  given  application  directly  from  design  engineering  information. 

•  Decision  Automation 

Automate  the  capture  and  combination  of  essential  elements  of  both  quantitative  and 
qualitative  system  models.  Attempt  to  automate  appropriate  decision-making  knowl¬ 
edge.  The  learning  apprentice  suggested  above  provides  an  appropriate  starting 
point. 


Engineering  Analysis 

Logistics  analysis  with  regard  to  maintenance  support  may  be  thought  of  as  consider¬ 
ing  particular  forms  of  man-machine  interaction.  This  heading  addresses  the  machine  com¬ 
ponent;  that  is,  analysis  focused  on  the  equipment  itself.  Engineering  analysis  involves 
such  notions  as  reliability  and  criticality  of  individual  components,  prediction  of  failure 
modes,  and  effects  to  the  system  as  a  whole. 
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In  the  future,  engineering  analysis  will  be  based  more  directly  on  original  engineering 
design  data,  and  will  draw  more  heavily  upon  qualitative  physics  techniques.  Identification 
of  maintainability  problems  should  occur  early  for  feedback  into  the  design  process,  per¬ 
haps  with  expert-system  advice  for  modification. 

Several  problems  arise  in  the  use  of  qualitative  physics  for  LSA  engineering  analysis. 

•  Teleology  —  developing  a  higher  of  level  understanding  of  component  purpose 
and  operational  theory. 

•  Spatial  Reasoning  —  simulation  and/or  encoding  of  human  mechanical  insight 
about  part  shape  and  function. 

•  Total  Economics  —  encompassing  costs  of  diagnosis,  repair,  training,  reason 
ing,  and  so  forth  within  a  common  framework  for  comparative  analysis  and 
trade-off. 

As  with  most  of  the  emerging  technologies  which  draw  upon  engineering  data,  design 
diagnosis  and  “debugging”  based  on  such  techniques  will  require  sophisticated  formal 
models  for  the  capture  and  analysis  of  design  rationale. 

Short  Term 

•  Algorithmic  and  Causality  Analyses 

Extend  existing  forms  of  failure  prediction  and  criticality  analysis,  using  quantitative 
product  data  and  current  statistical  and  simulation  techniques  to  establish  causality 
connections,  malfunction  effects,  and  diagnoses. 

Medium  Term 

•  Engineering  Design  Critic 

Develop  an  expert-system-based  design  “critic”  for  reliability  analysis  and  evaluation 
of  implied  maintenance  requirements. 

•  Qualitative  Predictions 

Begin  to  apply  qualitative  physics  reasoning  to  the  prediction  of  component  failures 
and  expected  effects,  both  local  and  global.  Use  the  results  to  guide  diagnosis  tree 
construction. 

Long  Term 

•  Model  Integration 

Integrate  and  represent  the  dependencies  between  physical  modeling  from  engineer¬ 
ing  graphics,  qualitative  physics,  and  Al-based  planning  models. 
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•  Alternative  Elaborations 

Develop  simulation  and  reasoning/analysis  support  for  “what  if’  questions  regarding 
alternative  engineering  design  paths. 

•  Engineering  Redesign  Agent 

Extend  the  expert  system  “design  critic”  to  an  active  redesign  agent/advisor  able  to 
suggest  engineering  workarounds  to  predicted  maintenance  problems. 


Task  Analysis 

Task  analysis  addresses  the  human  side  of  the  maintenance  equation.  The  primary 
goal  of  task  analysis  is  tne  generation  of  maintenance  task  plans  from  product  data.  Repre¬ 
sentative  elements  within  this  heading  include  proceduralization  of  specific  repair  tasks,  and 
analysis  of  manpower  and  skill-level  requirements. 

Computer-based  human-modeling  systems  for  task  simulation  and  performance  analy¬ 
sis  will  assume  a  greater  role  within  LSA.  One  such  system,  described  and  demonstrated 
at  the  first  workshop,  is  Jack,  a  3D  graphics  system  for  definition  and  animation  of  simu¬ 
lated  human  figures  (Badler,  1991).  Built  on  a  sophisticated  representation  for  articulated 
figures,  including  joint  constraints  and  strength  models,  Jack  offers  the  human  factors  en¬ 
gineer  or  ergonomics  analyst  a  wide  range  of  capabilities  for  human  performance  assess¬ 
ment.  The  DEPTH  program,  also  described  at  the  workshop,  intends  to  combine  the  best 
features  of  both  Jack  and  Crew  Chief,  another  well-known  human-modeling  system 
(Easterly  &  Ianni,  1991). 

Workshop  participants  suggested  a  variety  of  projects  directed  toward  the  general  goal 
of  increased  sophistication  in  human-modeling  systems.  Several  ideas  for  integrating  such 
simulations  more  directly  with  engineering  data  and  the  product  development  process  as  a 
whole  were  discussed.  Some  additional  suggestions,  such  as  animating  task  scenarios  di¬ 
rectly  from  natural  language  descriptions,  offer  intriguing  research  projects  for  future  hu¬ 
man-modeling  systems,  but  do  not  specifically  address  the  maintenance  task/planning  is¬ 
sue,  that  is,  the  extraction  of  maintenance  action  sequences. 

As  simulation  analysis  becomes  more  proceduralized  and  relies  less  on  human  obser¬ 
vation,  possibilities  for  directly  passing  task  “critiques”  back  to  engineering  or  task 
“descriptions”  forward  to  aid  the  authoring  process  can  begin  to  be  pursued.  Automated 
planning  will  play  a  significant  role  in  the  anticipated  progression  from  human  control  and 
analysis  of  task  animations  to  their  construction  and  evaluation  by  machine. 

Short  Term 

•  Visual  Extensions 

Pursue  the  inclusion  and  use  of  visual  factors  in  simulations,  such  as  where  the 
technician  looks  and  what  he  sees.  Incorporate  visual  cues  with  the  automatic  gen¬ 
eration  of  lighting  and  shadows  (radiosity).  Visual  planning  languages  could  poten¬ 
tially  serve  as  the  user  interface. 
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•  Language  Investigation 

Begin  construction  of  a  “standard  verb  list”  for  encoding  of  animation  actions.  Ana¬ 
lyze  existing  TOs  for  language  content  and  descriptive  style. 

Medium  Term 

•  Biomechanical  Enhancements 

Develop  a  complete,  biomechanically  accurate  model,  including  (a)  sophisticated 
strength  models  for  the  whole  body,  (b)  smooth  real-time  response,  (c)  collision 
avoidance,  and  (d)  dynamic  joint  constraints  with  acceleration  factors. 

•  Agent  Customization 

Build  a  behavior  “library”  and  customize  task  performance  to  specific  anthropometric 
data.  Investigate  technician  skill  levels  as  factors  in  task  performance. 

•  Language  Database 

Develop  computational  definitions  of  verbs  and  actions,  including  object  context  and 
use  of  adverbs.  Begin  definition  of  a  large-scale  world  database;  draw  upon  CAD 
data  to  generate  actions  from  object  connections. 

Long  Term 

•  Knowledge  Representation 

Complete  compilation  of  the  task  description  “verb  list.”  Extend  the  generation  of 
natural  language  descriptions  from  simulation  steps.  Dynamically  link  actions  to  text 
descriptions  (i.e.,  “learning”  a  new  verb  definition). 

•  Cognitive  Modeling 

Develop  profiles  of  technicians’  general  task  approaches/strategies  (e.g.,  “doer”  vs. 
“thinker”).  Associate  skill  level  and  body  factors  to  task  completion  failures.  Model 
the  learning  of  skills  and  associated  performance  improvements. 

•  Planning  and  Reasoning 

Add  technician  access  and  egress  planning.  Augment  equipment  models  with  com¬ 
mon-sense  reasoning  about  gravity  and  basic  physics.  Automate  the  evaluation  of 
alternative  task  execution  (posture  planning  and  movement  optimizations). 


Text  Generation  and  Graphics  Synthesis 

The  overall  goal  in  technical  authoring  is  increased  automation.  Augmenting  human 
writers  and  illustrators  with  systems  which  generate  maintenance  instructions  and  synthe¬ 
size  equipment  diagrams  will  provide  enormous  and  immediate  cost  benefits  and  raise  the 


23 


degree  of  customization  (e.g.,  different  repair  contexts,  technician  skill  levels,  etc.)  practi¬ 
cal.  This  subsection  addresses  the  creation  of  textual  instructions  for  diagnosis  and  repair 
procedures,  supporting  graphics,  and  general  descriptive  information,  as  well  as  their  link¬ 
age  through  various  forms  of  cross-referencing. 

The  potential  for  future  automation  in  this  area  involves  at  least  two  related  dimen¬ 
sions. 

•  Synthesis  of  text  and  graphics  directly  from  the  LSA  database  to  be  stored  in  an 
associated  “view  package”  database  for  loading  by  the  technician,  prior  to  begin 
ning  a  maintenance  task. 

•  Dynamic  generation  of  appropriate  technical  instructions  and  illustrations  during 
the  maintenance  task  itself. 

The  latter  dimension  may  be  alternately  viewed  as  blurring  the  distinction  between 
“authoring”  and  “presentation,”  or  as  promoting  the  authoring  task  to  a  meta-level.  That  is, 
rather  than  performing  this  task,  the  (meta-)author  would  specify  (in  software)  how  main¬ 
tenance  instructions  should  be  generated  from  product  knowledge. 

The  Coordinated  Multimedia  Explanation  Testbed  (COMET)  system,  demonstrated  at 
the  workshop,  is  one  example  of  current  research  work  in  this  area  (Feiner  &  McKeown, 
1990).  COMET  uses  a  hierarchical  knowledge  base  of  components  and  repair  procedures 
to  dynamically  plan  the  information  content  of  each  display  (McKeown  et  al.,  1991).  It 
then  selects  and  generates  appropriate  text  and  graphics  explanation  segments.  Bringing 
systems  like  COMET  out  of  the  research  laboratory  and  into  the  technician’s  PMA  will  re¬ 
quire  more  direct  use  of  product  information,  more  sophisticated  planning  components,  and 
further  advances  in  display  layout  and  user  interaction. 

As  equipment  becomes  more  complex  and  maintenance  instruction  more  specialized, 
formal  methods  for  establishing  the  correctness  of  both  information  content  and  presenta¬ 
tion  assume  a  greater  role.  The  electronic  manuals  of  the  future  will  not  only  be  interactive, 
but  also  highly  dynamic  and  capable  of  supporting  a  wide  range  of  possible  maintenance 
scenarios.  The  large  number  of  potential  repair  contexts  and  distinct  presentations  to  suit 
them  makes  manual  validation  (i.e.,  working  through  every  possible  scenario  by  hand)  in¬ 
feasible.  Automated  verification  systems  must  begin  to  take  over  verification/validation  re¬ 
sponsibilities  in  this  area. 

In  step  with  the  expected  automation  of  authoring,  the  focus  of  verification  technology 
will  likewise-  shift  from  the  technical  information  itself  to  the  automated  processes  which 
produce  it.  The  desired  certification  will  likely  depend  upon  establishing  both  correctness- 
preserving  properties  of  the  generation  processes,  and  consistency  requirements  within  the 
PKB  and  other  knowledge  sources. 

The  subsections  below  collect  proposed  generation  and  synthesis  goals  according  to 
their  respective  identification  with  either  procedural  or  general  descriptive  information.  As 
noted  earlier,  diagnostic  information  was  not  a  focus  of  the  workshops. 
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Procedural  Information 


The  stepwise  detailing  of  repair  and  test  procedures  is  action -oriented  information, 
which,  when  “processed”  by  the  technician,  effects  changes  in  equipment  status.  The  im¬ 
plied  dynamic  effects  of  presenting  such  information  must  be  modeled  within  the  genera¬ 
tion  system. 

Short  Term 

•  Stepwise  Decomposition 

Develop  techniques  for  drawing  task  decompositions  from  the  human-modeling 
simulation  database.  Use  timing,  position,  and  posture  information  to  identify  natu¬ 
ral  task  breakpoints. 

•  Plan  Annotation 

Use  maintenance  planning  output  (a  plan  data  structure)  from  the  logistics  analysis  as 
a  structural  “template”  for  annotation  by  a  human  expert.  Build  workstation-like 
tools  to  support  the  user  in  filling  out  this  template  with  English  text. 

•  Behavioral  Certification 

Formulate  logical  assertions  regarding  stepwise  presentation  of  tasks,  satisfaction  of 
preconditions,  handling  of  follow-on  conditions,  and  so  forth  within  a  “populated” 
Interactive  Electronic  Technical  Manual  (DETM)  database.  Build  a  corresponding  re¬ 
active  model  and  use  an  inference  engine  to  certify  correct  behavior  by  proving  sat¬ 
isfaction  of  these  assertions. 

Medium  Term 

•  Causality  and  Conditionals 

Represent  causality  and  effects  relationships  within  the  maintenance  task  database. 
Generate  input  conditions,  post-conditions,  and  follow-on  conditions  from  higher- 
level  connections. 

•  Coordination  Constraints 

Incorporate  interacting  constraints  between  generators  for  different  media  (e.g.,  text 
and  graphics)  which  enforce  correspondence  in  the  underlying  structure  (e.g.,  three 
simple  sentences  related  to  three  subdiagrams). 

•  Temporal  Media 

Explore  authoring  for  alternative  time-ordered  media  such  as  animation  and  speech. 
Model  temporal  relations  within  the  content  planner  for  generation  and  synthesis  to 
manage  phrasing,  delay,  and  so  forth. 
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Long  Term 


•  Plan-Based  Synthesis 

Generate  textual  instructions  directly  from  plan  data  structure(s).  Determine  whether 
sufficient  information  is  present  to  support  this. 

•  Stylistic  Variations 

Support  a  variety  of  style  conventions  within  text  and  illustration  generators.  Select 
and  alter  them  based  on  display  hardware  profiles,  human  factors  guidelines,  and 
user  preferences. 

•  Verified  Transformations 

Formally  establish  “correctness-preserving”  properties  of  the  generation  and  synthe¬ 
sis  systems  used  to  automate  the  production  of  text  and  graphics  from  LSA  sources. 


Descriptive  Information 

Technical  data  not  specific  to  a  particular  maintenance  procedure  (e.g.,  operational  ex¬ 
planations,  generic  illustrations,  and  various  forms  of  cross-referencing)  fall  in  the  category 
of  descriptive  information.  Since  the  intended  usage  will  be  less  predetermined  than  that  of 
procedural  information,  the  content  will  be  more  dynamic. 

Short  Term 

•  Simple  Animations 

Animate  two-dimensional  illustrations  of  component  modules  and  indicate  the  data 
flow  along  interconnections.  Develop  standard  mechanisms  to  render  and/or  high¬ 
light  individual  components  based  on  their  current  fault  classification. 

Medium  Term 

•  Design-Based  Illustration 

Generate  3D  equipment  illustrations  from  engineering  CAD  models.  Support  ex¬ 
ploded  views,  and  include  cross-references  to  textual  operation  descriptions. 

Long  Term 

•  Complex  Animations 

Generate  3D  animations  from  design  models  and  qualitative  physics  information. 
Show  internal  details  and  mechanical  interaction.  Provide  “cut-away”  illustrations, 
interactive  zooming,  view  rotations,  and  other  alternative  viewpoints. 
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Dynamic  Presentation  and  Virtual  Reality 

Maintenance  presentation  covers  the  actual  delivery  of  information  to  the  technician 
during  a  given  maintenance  task.  This  delivery  must  be  dynamically  driven  by  the  related 
tasks  of  diagnosing  and  resolving  problems.  In  the  near  term,  this  information  will  consist 
of  a  sequence  of  interactive  electronic  displays  appearing  on  the  technician’s  PMA.  The 
screens  presented  during  a  given  maintenance  scenario  will  vary  greatly  with  the  techni¬ 
cian’s  actions  and  the  ongoing  repair. 

As  generation  and  synthesis  tools  become  more  advanced,  and  PMAs  become  more 
powerful,  less  of  the  text  and  graphics  which  make  up  each  screen  is  likely  to  be  “canned” 
(i.e.  authored  and  stored  within  a  previously  loaded  view  package)  and  more  is  likely  to  be 
generated  dynamically.  A  subset  of  the  PKB,  along  with  the  generation  software,  could  be 
loaded  and  carried  within  the  PMA,  rather  than  a  (fixed)  number  of  precomposed  screens. 

As  the  displays  become  more  sophisticated  and  the  presentations  more  dynamic,  the 
notion  of  individual  (discrete)  screens  will  fade;  a  VR  presentation  appears  continuous.  As 
discussed  in  the  previous  section,  the  possibility  of  generating  text  and  graphics  “on  the 
fly”  (i.e.,  from  a  knowledge  base  of  maintenance  plans,  engineering  data,  and  so  forth)  ef¬ 
fectively  shifts  the  current  distinction  between  authoring  and  presentation  up  a  level  of  ab¬ 
straction  and  control.  Thus,  the  writers  write  parts  of  the  generation  software. 

The  advent  of  VR  systems  will  drastically  change  the  current  conception  of  electronic 
information  and  how  it  is  displayed.  Hardware  and  software  advances  in  this  realm  are 
rapidly  increasing  the  potential  for  the  practical  application  of  VR  to  a  wide  range  of  infor¬ 
mation  delivery  requirements.  It  was  claimed  during  the  workshop  that  within  ten  years, 
VR  capabilities  greatly  exceeding  those  available  today  could  be  supplied  through 
lightweight  electronic  “glasses”  and  a  “belt-pack”  computer. 

Managing  the  vast  databases  of  drawing  and  behavior  detail  required  is  a  significant  is¬ 
sue  for  VR-based  presentation.  The  engineering  design  information  stored  within  the  PKB 
could  be  used  more  or  less  direcdy  by  the  VR  software  to  generate  the  detail  necessary  for 
realistic  equipment  illustrations  as  overlays  or  perhaps  exploded  views.  However,  a  VR 
presentation  would  be  even  more  dynamic,  with  an  effectively  infinite  number  of  possible 
“scenarios,”  than  those  anticipated  for  the  coming  generation  of  PMAs.  Automated  means 
for  checking  and  certifying  such  presentations  or,  more  directly,  the  generation  software, 
will  be  even  more  necessary. 

The  first  subsection  below  addresses  identifying  and  solving  problems.  The  second 
subsection  completes  the  loop  of  passing  maintenance  experiences  in  the  field  back  to  LSA 
and,  ultimately,  engineering  design. 


Troubleshooting,  Replacement,  and  Repair 

The  basic  thrust  of  advances  in  troubleshooting,  replacement  and  repair  is  toward  in¬ 
creasing  the  technician’s  information  sources  through  multimedia  representations  of  equip¬ 
ment  state  and  potential  problems.  A  wide  variety  of  correlations  between  components’ 
physical  characteristics  and  possible  depictions  could  be  explored.  Temperature  shown  as 
color  and  current  depicted  via  tone  frequency  are  some  obvious  correlations.  Human  pat- 
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tern-matching  skills  are  very  good;  therefore,  the  technician  might  recognize,  through  such 
depictions,  some  mechanical  behavior  difficult  to  capture  algorithmically. 

The  technician’s  physical  relation  to  the  aircraft  is  of  more  concern  in  this  area  than  the 
others  because  subassemblies  may  be  unexpectedly  difficult  to  access.  Any  “contortions” 
required  of  the  technician  will  be  harder  to  describe  than  equipment  details.  Animations 
from  human-modeling  could  be  played  back  on  a  high-resolution  PMA  screen.  Ironically, 
dependence  on  the  PMA  can  limit  the  technician's  agility.  That  is,  holding  or  viewing  a 
typical  (even  small)  portable  “computer  with  display”  may  be  difficult  during  some  tasks. 
A  VR  headset  would  be  more  flexible  in  such  cases. 

Short  Term 

•  Selective  Scaling 

Begin  development  of  selective  scaling.  Investigate  highlight  and  zoom  mechanisms 
(e.g.,  cams,  cones,  abstractions,  fish-eye  views,  etc.)  for  functional  diagrams  and 
wiring  schematics.  Support  selective  emphases  according  to  current  fault/symptom 
states. 

•  Plan  Execution 

Implement  a  prototype  plan  execution  system  within  the  PMA.  The  plan  would 
come  from  LSA,  with  English  annotations,  and  be  highly  conditional.  Early  techni¬ 
cian  acceptance  is  critical. 

Medium  Term 

•  Audio  Output 

Investigate  ways  to  utilize  audio  output  as  a  diagnostic  aid,  such  as  listening  to  the 
current  draw  of  a  solenoid  through  an  earphone  for  which  characteristic  tone  changes 
indicate  impending  failure. 

•  History  Visualization 

Construct  visual  representations  of  component  performance  history  (e.g.,  graph,  bar 
chart,  color  coding,  etc.)  for  potential  diagnosis  by  the  technician. 

•  Virtual  Overlays 

Provide  VR  capabilities  for  highlighting  suspected  components  or  subsystems  as 
overlays  viewed  within  a  “heads-up”  display.  Offer  these  capabilities  across  the 
whole  (or  most  of)  a  selected  vehicle/system. 

•  VR  Toolkit 

Support  technician  selection  of  specific,  modular  VR  capabilities  and/or  equipment 
visualization  data  structures  (sub-“worlds”)  as  “tools”  for  a  particular  repair  task. 
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•  Dynamic  Replanning 

Provide  a  limited  PMA  replanning  capability  making  (probable)  use  of  temporal 
logic.  Determine  if  this  is  a  desired  capability. 

Long  Term 

•  Behavior  Animations 

Develop  VR  animations  simulating  behavior  and  technician  observation  of  equipment 
with  various  possible  problems.  Generate  the  animations  dynamically  from  test  re¬ 
sults  and  equipment  self-monitoring  information. 

•  Subassembly  Detailing 

Generate  holographic  or  VR  depictions  of  individual  component  fit,  access,  assem¬ 
bly,  and  disassembly  within  complex  mechanical  substructures.  Investigate  uses  of 
color,  animation,  rotations,  alternate  view  points,  and  so  forth. 


Capture  and  Review 

A  variety  of  new  knowledge  may  be  gained  during  the  actual  execution  of  a  given 
maintenance  activity.  The  recording  and  linking  of  this  knowledge  back  into  LSA  pro¬ 
cesses  for  review  offers  an  additional  dimension  for  maintenance  support  analyses. 

Short  Term 

•  History  Replay 

Replay  maintenance  session  history  to  check  for  inconsistencies  and  compare  with 
statistical  projections  and  certification  results. 

Medium  Term 

•  Decision  Support  Feedback 

Record  equipment  failure  rates  and  maintenance  operation  problems  within  the  PMA 
for  uploading  back  to  LSA  decision-support  components.  Attempt  to  facilitate  better 
decision-making  in  the  future. 

Long  Term 

•  VR  Recording 

Record  actual  repair  operations  performed  by  the  technician  through  a  “data-glove.” 
Investigate  use  of  a  full  VR  “body-suit”  for  task  recording.  Evaluate  whether  this  is 
too  awkward  and/or  constricting.  Compare  the  results  with  human-modeling  simu¬ 
lations  for  modification  and  re-analysis. 
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V.  CONCLUSIONS  AND  REVIEW 


PMAs,  sophisticated  task  animations,  and  the  possibility  of  automatically  generating 
instructional  text  and  illustrations  were  unknown  13  years  ago  when  the  “Sgt.  Bayshore” 
scenario  was  written.  The  coming  decade  holds  a  similar  promise  for  VR,  comprehensive 
and  accessible  product  databases,  and  expert  system  and  domain-specific  planning  capa¬ 
bilities  expected  to  become  practically  available.  To  realize  that  promise,  similar  plans  for 
extending  and  integrating  these  emerging  technologies  must  be  formulated  now. 

As  the  first  step  towards  developing  such  a  plan,  AL/HRG  hosted  a  workshop  in  Palo 
Alto  which  brought  together  a  number  of  experts  to  discuss  current  and  future  work  in 
various  applicable  fields. 


Key  Themes 

Several  primary  themes  for  a  future  focus  emerged  from  the  first  workshop: 

•  Comprehensive  Product  Information — All  design,  engineering,  and 
logistics  information  must  be  kept  in  a  common  PKB  throughout  a  system's  or 
component’s  effective  life  cycle.  Information  should  be  stored  and  maintained  in 
accordance  with  standard  organizing  principles,  and  be  supported  with  standard 
mechanisms  for  extraction  and  review  across  a  wide  range  of  potential  users,  both 
human  and  machine. 

•  Automated  Generation  of  TOs — Automated  synthesis  tools  can  begin  to 
augment,  and  eventually  replace,  human  writers  and  illustrators.  The  “authoring” 
task  should  become  one  of  applying  and  tailoring  these  tools  to  produce  the  desired 
output. 

•  Formal  Certification  Methods — Traditional  “validation  and  verification” 
procedures  must  be  supplanted  by  automated  certification  tools.  Both  the  depth  of 
technical  information  and  variability  of  its  presentation  will  make  manual  checking 
of  consistency  and  correctness  infeasible. 

•  New  Presentation  Capabilities — Electronic  presentation  of  text  and 
graphics  on  a  PMA  will  seem  primitive  compared  to  the  coming  possibilities  for  in¬ 
formation  delivery.  The  hardware  for  supplying  virtual  overlays  on  a  head- 
mounted  visor  (for  instance)  or  sophisticated  3D  animations  is  imminent.  The 
software  for  extracting  the  necessary  engineering  detail  to  utilize  this  potential 
should  be  developed  now. 

•  Design  Concurrency — Technology  must  be  developed  for  controlling  and 
implementing  design  and/or  operational  changes  from  the  CAD/CAE  workstation, 
through  technical  data  generation,  to  delivery  in  the  field.  This  is  fundamentally  a 
Computer  Aided  Logistics  Support  (CALS)  issue.  Accuracy  and  currency  of  tech¬ 
nical  data  must  be  assured,  no  matter  how  it  is  delivered,  because  systems  are  con¬ 
stantly  being  reconfigured  or  re-engineered. 
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VI.  RESEARCH  RECOMMENDATIONS 


The  first  workshop  laid  the  groundwork  for  developing  future  maintenance  support 
capabilities  and  identified  the  primary  research  directions  and  key  technologies  to  be  pur¬ 
sued.  The  second  workshop  refined  this  focus  and  defined  an  initial  set  of  capabilities 
upon  which  work  could  potentially  begin. 

The  subset  of  original  participants  who  reconvened  at  AL/HRG  in  March  1992  were 
instructed  to  narrow  their  focus  considerably  in  order  to  agree  on  a  specific  research  agenda 
for  prospective  funding.  Accordingly,  the  panel  decided  to  concentrate  on  the  mid-phase 
processes  of  LSA  and  Technical  Authoring.  These  activities,  particularly  the  creation  and 
validation  of  technical  data,  have  traditionally  been  the  most  expensive  in  terms  of  human 
resources. 

The  group’s  working  premise  was  that  various  forms  of  product  information  would  be 
available  from  engineering  phases  and,  similarly,  various  electronic  presentation  capabili¬ 
ties  would  be  developed  for  information  delivery.  Focusing  on  the  middle  phases  of 
LSA/authoring  would  help  to  better  define  both  the  specific  product  data  required  from  en¬ 
gineering  and  the  general  structure  of  the  resulting  maintenance  task  information. 

Within  this  restricted  scope,  a  number  of  individual  subprocesses  were  identified.  The 
panel  agreed  that  it  was  essential  to  compose  an  integrated  collection  of  prototype  tools  that 
would  cut  across  more  than  one  maintenance  (sub-)product  phase.  The  suggested  high- 
level  approach  was  to  select  one  or  more  relatively  simple  components  (such  as  a  pump  or 
radar  display)  to  provide  a  point-to-point  focus  for  the  individual  projects.  This  concentra¬ 
tion  would  facilitate  demonstration  of  the  passing  forward  (and  back)  of  information  be¬ 
tween  different  activities,  again  recognized  as  a  cornerstone  of  the  planned  advances. 


Architectural  Design 

To  provide  a  skeletal  framework  for  researchers  developing  individual  tools,  the  group 
made  several  high-level  architectural  design  decisions.  The  accommodation  and  integration 
of  diverse  technical  information,  expressed  within  a  variety  of  representations,  and  the  po¬ 
tential  for  inconsistency  and  error  which  naturally  accompanies  such  diversity,  were  the 
key  points  to  be  addressed. 

The  stepwise  planning  and  detailing  of  aircraft  maintenance  tasks  involve  a  number  of 
different,  though  interrelated,  forms  of  information.  Engineering  geometry,  human  perfor¬ 
mance  models,  textual  instructions,  and  graphic  illustrations  are  primary  examples  of  the 
distinct  data  domains  which  must  somehow  be  coordinated  to  maintain  overall  consistency 
and  correspondence. 

Since  maintenance  plans  are  both  the  foremost  product  of  LSA  and  the  key  element  of 
support  information,  the  group  agreed  that  a  planning  language  representation  should  pro¬ 
vide  the  core  knowledge  formulation  and  structural  foundation  for  the  maintenance  knowl¬ 
edge  base.  Engineering  geometry,  performance  models,  text,  illustrations,  and  other  forms 
of  information  will  be  attached  to  and  organized  within  the  internal  plan  data  structures. 
Conceptually  surrounding  this  plan-based  core  are  a  number  of  relatively  independent  tools 
which  provide  services  such  as  extraction,  explanation,  and  consistency  checking,  specific 
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Electronic  Technical  Manual  Generation:  Functional  Architecture  View 


to  the  other  information  domains.  Figure  4  illustrates  the  basic  functional  architecture  and 
introduces  the  principal  tools  to  be  developed. 
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Figure  4 

Proposed  Functional  Architecture  and  Principal  Tools 


Planning  Language  Foundation 

A  uniform  plan  representation  is  central  to  the  design  of  the  maintenance  information 
knowledge  base.  Figure  5  shows  a  basic  Al-planning  view  of  the  TO  generation  process. 
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The  planning  language  selected  (or  developed)  should  be  hierarchical  in  order  to  permit 
incremental  decomposition  in  accordance  with  aircraft  system  and  component  hierarchies, 
and  down  through  individual  maintenance  task  steps.  As  the  foundation  for  the  wide  range 
of  technical  information  anticipated,  the  planning  language  must  be  flexible  and  extensible.  It 
must  accommodate  the  variety  of  supporting  data  currently  identified  as  well  as  those  to  be 
developed  in  the  future. 

The  various  forms  of  additional  information  will  presumably  be  stored  within  the 
database  through  a  corresponding  variety  of  annotations  to  the  basic  plan  structure.  Further 
annotations  could  state  dependency  relationships,  specify  constraints  to  be  maintained, 
and/or  assert  conditions  expected  to  hold  among  these  annotations.  Thus,  the  planning 
language  base  should  further  support  notational  facilities  for  expressing  such  connections 
and  inference  mechanisms  for  reasoning  about  them. 

The  central  role  of  the  planning  language  dictates  that  one  language  be  settled  upon  as 
soon  as  possible,  since  design  of  all  the  other  tools  will  depend  on  (at  least)  its  high-level 
structure.  This,  in  turn,  suggests  that  an  existing  language  which  best  satisfies  the  criteria 
above  should  be  chosen,  rather  than  designed  and  developed  from  scratch,  thus  forcing  the 
delay  of  significant  work  on  other  tools  until  its  details  emerge.  The  Reacto  specification 
language  and  verification  system  was  discussed  by  the  group  as  an  example  of  the  kind  of 
extensible  foundation  sought  (Gilham  et  al.,  1989). 

The  Reacto  language,  designed  for  the  specification  and  verification  of  complex  reac¬ 
tive  systems,  is  formally  based  on  the  model  of  hierarchical  finite  state  machines  or  State- 
Charts  (Hard,  1987).  A  state-chart  specification  may  be  viewed  as  corresponding  to  a  plan 
in  which  the  universe  of  possible  state  descriptions  has  been  recursively  partitioned  into  a 
tree  of  state  classifications.  Reacto's  existing  graphic  interface  for  specification  acquisition, 
automated  verification  capabilities,  and  interactive  simulation  facilities  could  be  enhanced 
and/or  modified  to  similar  LSA  and  maintenance  information  requirements. 

Some  other  Al-based  planning  systems  were  also  mentioned  as  potential  candidates. 
The  Noah  system  has  been  applied  to  the  planning  of  a  pump  overhaul  task  (Sacerdoti, 
1975)  and  the  O-Plan  project  has  addressed  some  aspects  of  interactive  plan  animation 
(Currie  &  Tate,  1991).  A  careful  comparison  of  the  inherent  advantages  and  potential 
problems  of  existing  candidates  must  be  made,  as  well  as  their  likely  ease  of  adoption.  The 
system  selected  will  exert  considerable  influence  on  the  research  and  development  of  auto¬ 
mated  translation  tools,  design  critics,  and  domain-specific  assistants  which  collectively 
serve  the  planning  domain  and  its  various  subordinates. 


Translators,  Critics,  and  Assistants 

As  introduced  above,  technical  information  from  other  data  domains  will  also  be  in¬ 
corporated  within  the  overall  maintenance  plan  hierarchy.  Though  some  functional  overlap 
may  be  expected  and  hoped  for,  in  terms  of  an  architectural  overview,  it  is  simplest  to  as¬ 
sume  that  each  such  domain  (e.g.,  engineering  geometry,  human  models,  text,  graphics, 
etc.)  will  be  supported  by  separate  and  independent  tools  constructed  specifically  to  serve 
the  associated  information  type. 

Nonetheless,  the  individual  tools  may  be  viewed  as  sharing  several  basic  operational 
forms,  in  addition  to  their  common  knowledge  concerning  both  the  high-level  structure  of 
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plans  and  whatever  standard  protocol  exists  for  communicating  with  the  plan  database.  In 
general,  each  domain  will  require  facilities  to  support:  (a)  storage  and  retrieval,  (b)  evalua¬ 
tion  and  consistency  checking,  and  (c)  user-  and/or  author-directed  modification  of  techni¬ 
cal  data  within  the  given  domain.  Accordingly,  the  tools  to  be  developed  may  be  catego¬ 
rized  with  respect  to  their  basic  function  as  “translator,”  “critic,”  or  “assistant.” 

Translation  tools  support  information  flow  across  database  boundaries,  by  bridging 
the  gap  between  plan  data  structures  and  the  alternative  representations  specific  to  the  other 
domains.  This  “translation”  may  in  some  cases  simply  be  the  accessing  of  a  particular  an¬ 
notation,  such  as  an  illustration  accompanying  a  task  step.  Other  cases  may  involve  signifi¬ 
cant  analysis  and  processing,  such  as  the  incorporation  or  extraction  of  numerical  data  relat¬ 
ing  physical  exertion  and  expected  task  performance  time. 

Domain- specific  “critics”  judge  and  report  on  the  relative  “goodness”  of  database  in¬ 
formation.  Consistency  checking  and  maintenance  is  the  most  important  category  for  such 
evaluation,  but  there  are  other  possible  categories  as  well.  For  example,  illustrations  can  be 
graded  as  more  or  less  cluttered  and  instructions  as  more  or  less  readable  independent  of 
die  degree  to  which  they  concur  with  other  technical  data. 

The  potential  autonomy  of  such  tools  covers  a  broad  spectrum.  At  one  end  lie  static 
search  procedures  and/or  pattern-matchers,  invoked  by  the  user/analyst  and  given  a  specific 
condition  to  check  for.  A  number  of  knowledge-base  support  systems  provide  this  basic 
capability.  At  the  other  end,  one  may  envision  active  agents  which  continuously  monitor 
changes  within  the  database  for  violation  of  interdependencies  and  functional  constraints, 
infer  the  additional  modifications  required,  and  automatically  perform  them.  Such  agents 
have  been  implemented,  for  example,  within  the  Project  Management  Assistant  to  address  a 
wide  range  of  complex  ( condition ,  action  )  pairs  involving  event  characterizations,  policy 
descriptions,  and  a  variety  of  constraint  formalisms  (Qian  et  al.,  1990). 

The  notion  of  domain  “assistants”  follows  traditional  usage  of  the  term  in  suggesting 
(a)  an  integrated  collection  of  high-level  capabilities  for  user  interaction,  (b)  some 
“understanding”  of  the  domain’s  internal  relationships,  and  (c)  the  automation  of  more  or 
less  mechanical  subtasks  (Green  et  al.,  1993).  As  in  other  realms,  the  basic  user  task  to  be 
supported  is  the  creation  and  incremental  refinement  of  some  formal  description(s),  in  this 
case  varieties  of  logistics  information  and  maintenance  task  plans,  along  with  instructions 
and  illustrations  for  the  technician.  Layered  on  top  of  other  interactive  tools,  the  assistant 
will  normally  draw  upon  and  may  totally  encompass  the  functionality  outlined  above. 

Depending  on  the  complexity  of  a  given  domain  and  the  current  degree  to  which  its 
processes  are  formally  understood  and  have  been  proceduralized,  the  assistant’s  contribu¬ 
tion  may  range  from  simply  improving  user  efficiency  to  assuming  most  of  the  attendant 
task.  In  text  authoring,  for  example,  interactive  editors  will  eventually  be  supplanted  by 
assistants  which  synthesize  textual  instructions  directly  from  the  maintenance  plans,  requir¬ 
ing  only  minimal  guidance  and  little  editing/review  by  the  human  “author.” 

While  not  all  the  tools  proposed  in  the  research  agenda  below  fit  neatly  into  these  three 
categories  (translator,  critic,  or  assistant),  many  of  them  do.  The  exposition  above  hope¬ 
fully  offers  some  insight  for  uniform  tool  design. 


Initial  Agenda 

The  project  items  outlined  below  constitute  an  initial  research  agenda  for  the  next  one 
to  two  years.  As  noted  above,  the  intention  is  to  select  a  particular  component  or 
subassembly  (such  as  a  pump),  and  use  it  to  help  focus  and  connect  potential  advances 
across  the  typical  phases  of  maintenance  information  support — from  engineering  design  to 
presentation.  The  items  below  concentrate  primarily  on  LSA/authoring  processes,  but 
other  processes  are  addressed  (to  some  extent)  as  well  to  emphasize  and  demonstrate  at 
least  minimal  point-to-point  connections. 


(I)  Engineering  Feature  Extraction 

Extract  several  key  engineering  features  from  the  product  design  for  the  selected 
component(s)  and  incorporate  them  within  the  planning  knowledge  base.  This  initial 
exercise  in  design  recovery  will  be  done  largely  by  hand,  but  may  also  employ  avail¬ 
able  translation  capabilities  (see  Item  3)  outlined  below.  The  intent  is  to  draw  upon 
CAD  and  other  (e.g.,  PDES)  defining  data  to  help  identify  the  additional  extraction 
and  translation  facilities  required,  and  to  begin  to  address  the  use  of  qualitative 
physics  in  logistics  analysis. 


(2)  Geometric  Model  Construction 

Construct  one  or  more  geometric  models  to  support  maintenance  planning  and  simu¬ 
lation  for  the  product(s)  addressed.  The  models  should  link  with  existing  DEPTH 
facilities  and  also  be  accessible  through  the  planning  representation  developed  as  part 
of  this  agenda.  As  with  Item  1,  construction  of  specific  models  for  the  selected  sub- 
assemblies  will  serve  to  expose  the  foundations  for  eventually  automating  such  con¬ 
structions  and  will  help  to  focus  future  efforts  in  this  area. 


(3)  Prototype  Translation  Facilities 

Develop  a  preliminary  set  of  facilities  for  translating  engineering  design  descriptions 
into  corresponding  plan-based  representations.  These  facilities  should  augment  and 
be  developed  concurrently  with  the  (mostly)  manual  extraction  suggested  in  Item  1. 
Emerging  PDES  standards,  current  IETM  data  model  specifications,  and  external 
plan  formalisms  are  the  principal  domains  to  be  considered. 


(4)  Device  Engineering  Critic 

Design  and  implement  an  initial  device  engineering  critic  to  address  maintenance 
implications  of  product  design.  This  component  will  access  and  analyze  engineering 
data  incorporated  within  the  planning  knowledge  base,  and  draw  upon  its  own  engi¬ 
neering  knowledge  base  to  provide  feedback  to  both  maintenance  task  developers 
and  product  engineers.  Such  information  might  also  be  passed  back  to  design  pro¬ 
cesses  directly,  through  translation  (as  in  Item  3)  to  the  appropriate  engineering  rep¬ 
resentation. 
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(5)  Basic  Procedure  Library 


Define  the  structure  and  organization  of  a  basic  maintenance  procedure  library.  The 
structure  of  procedure  descriptions  and  how  they  are  indexed  and  classified  must  be 
established,  using  the  planning  representation  as  a  base.  The  library  must  then  be 
populated  with  25  to  30  standard  maintenance  actions,  using  composition  tools  (such 
as  Item  6)  when  possible.  Serving  as  an  extension  of  the  planning  knowledge  base, 
such  a  library  will  be  essential  for  procedure  uniformity  and  reuse. 


(6)  Procedure  Assembler 

Develop  an  interactive  tool  for  assembling  and  composing  maintenance  procedures 
from  individual  steps  and/or  subprocedures.  The  tool  must  support  basic  access  and 
storage  capabilities  across  the  procedure  library,  as  well  as  facilities  for  construction 
and  modification  of  procedure  descriptions,  perhaps  through  the  provision  of  proce¬ 
dure  templates.  In  addition,  the  tool  should  help  insure  that  hierarchical  correspon¬ 
dence  with  the  underlying  maintenance  plan  be  upheld  and  assist  with  simple  forms 
of  condition  propagation  and  consistency  checking  between  individual  steps. 


(7)  Procedure  Critic 

Design  and  implement  a  preliminary  procedure  critic  to  analyze  and  evaluate  descrip¬ 
tions  within  the  library.  Together  with  Item  6,  this  will  provide  a  foundation  for  de¬ 
veloping  an  integrated  procedure  assistant.  Procedure  principles  to  be  addressed  in¬ 
clude  basic  structural  requirements,  precondition  achievement,  potential  policy  viola¬ 
tions,  pre-  and  post-condition  matching,  optimization,  and  causality  effects. 


(8)  Planning  Assistant 

Explore  the  potential  for  procedure  synthesis,  through  the  development  of  a  proto¬ 
type  planning  assistant.  Though  potentially  expensive,  successful  implementation  of 
such  an  agent  would  provide  a  very  high  payoff,  and  largely  subsume  Items  6  and  7. 
The  idea  is  to  approach  procedure  construction  and  evaluation  from  a  higher-level 
planning  viewpoint,  employing  such  notions  as  resource  minimization,  optimization, 
generalization,  and  plan  reuse.  Explicit  maintenance  procedure  descriptions  would 
be  generated  automatically  from  the  annotated  plans. 


(9)  Text  Generation  Assistant 

Apply  and  extend  current  capabilities  for  automated  text  generation  within  a  (more 
comprehensive)  text  generation  assistant.  The  basic  synthesis  approach  will  employ 
templates  corresponding  to  the  given  procedure  decomposition  and  standard  word 
lists.  Initial  output  may  be  somewhat  stilted  but  will  improve  with  the  incorporation 
of  human  factors  constraints.  As  with  Item  6,  the  templates  should  be  linked  to  the 
appropriate  elements  of  the  plan  representation  to  facilitate  evaluation  and  consis¬ 
tency  checking. 
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(10)  Layout  Assistant 


Develop  a  layout  assistant  to  combine  text,  tables,  and  graphics  within  technical  pre¬ 
sentations.  The  presumed  capabilities  (previously  demonstrated)  include  extraction 
and  scaling  of  equipment  illustrations  from  engineering  (CAD)  drawings,  synthesis 
of  simple  block  diagrams  from  functional  abstractions,  and  generation  of  tables  from 
relational  data.  Stylistic  variations  and  structural/stepwise  correspondence  between 
text  and  related  graphics  should  also  be  supported. 


(11)  Screen  Structure  Critic 

Design  and  implement  a  prototype  critic  for  the  presentation  of  maintenance  instruc¬ 
tions  as  distinct  PMA  screen  displays.  This  is  a  first  step  towards  automating  the  fi¬ 
nal  QA  process,  currently  performed  by  humans.  One  initial  approach  would  be  to 
develop  a  set  of  metrics  for  screen  structure  and  layout,  and  use  them  to  tag  ques¬ 
tionable  displays  for  human  review.  Since  these  electronic  "pages"  potentially  vary 
with  the  presentation  hardware,  as  do  guidelines  for  complexity  with  technician  skill 
levels,  the  design  should  anticipate  parameterization  of  the  evaluation  metrics. 


(12)  Consistency  Checking  Engine 

Begin  to  identify  and  collect  the  inference  and  analysis  mechanisms  (used  by  various 
“critic”  agents)  into  a  generic  consistency  checking  engine.  Issues  to  be  addressed 
include  correspondence  between  plans  and  procedures,  resource  and  time  con¬ 
straints,  test  information,  configuration  management,  change  propagation,  and  ver¬ 
sion  control.  The  idea  is  to  eventually  provide  a  general-purpose  automated  reason¬ 
ing  server  to  support  the  particular  analyses  required  across  the  various  domains. 


(13)  Procedure  Interpreter! Animator 

Implement  a  basic  facility  for  interpreting  maintenance  procedure  descriptions  and 
animating  their  performance  by  a  technician.  Much  of  the  necessary  human-model¬ 
ing  and  graphics  technology  for  this  currently  exists  in  systems  such  as  Jack,  Crew 
Chief,  and  DEPTH.  Thus,  the  task  is  to  select  an  appropriate  subset  and  augment  or 
modify  it  to  use  the  central  planning  representation  and  associated  procedure  de¬ 
scriptions,  as  required. 


(14)  Integrated  User  Interface 

Integrate  the  tools  and  capabilities  outlined  above  within  a  supporting  framework  for 
(graphical)  user  interaction.  This  amounts  to  a  working  prototype  of  the  envisioned 
author’s  workstation.  Individual  windows  will  display  both  static  and  dynamic 
views,  corresponding  to  the  different  represe  nation  domains,  of  the  maintenance 
procedure  under  construction.  Figure  6  illustrates  the  concept,  showing  a  plan  view, 
a  text  view,  a  presentation  system  view,  and  a  task  animation  view.  Intended  to  be  a 
relatively  low-cost  item,  this  demonstration  prototype  will  nonetheless  be  important 
for  obtaining  early  user  feedback  and  achieving  overall  program  acceptance. 


Presentation  System  Uieui  Window 


Message  Window 


Figure  6 

Integrated  Graphical  User  Interface — Author’s  Workstation  Concept 
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was  presented  the  Grace  Murray  Hopper  Award  by  the  ACM  in  1985.  He  received  the 
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of  graphical  displays,  reactive  system  modeling  and  verification,  and  formal  approaches  to 
software  testing. 
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